Document Type

Journal Article

Department/Unit

Department of Mathematics

Abstract

We study behavior of the restricted maximum likelihood (REML) estimator under a misspecified linear mixed model (LMM) that has received much attention in recent genome-wide association studies. The asymptotic analysis establishes consistency of the REML estimator of the variance of the errors in the LMM, and convergence in probability of the REML estimator of the variance of the random effects in the LMM to a certain limit, which is equal to the true variance of the random effects multiplied by the limiting proportion of the nonzero random effects present in the LMM. The asymptotic results also establish convergence rate (in probability) of the REML estimators as well as a result regarding convergence of the asymptotic conditional variance of the REML estimator. The asymptotic results are fully supported by the results of empirical studies, which include extensive simulation studies that compare the performance of the REML estimator (under the misspecified LMM) with other existing methods, and real data applications (only one example is presented) that have important genetic implications.

Publication Year

2016

Journal Title

Annals of Statistics

Volume number

44

Issue number

5

Publisher

Institute of Mathematical Statistics

First Page (page number)

2127

Last Page (page number)

2160

Referreed

1

Funder

Supported in part by the NSF Grants DMS-08-09127, SES-1121794, and the NIH Grant R01-GM085205A1; the NIH Grant R01-GM59507;the NSF Grants DMS-11-06690 and DMS-1407530; Grant #61501389 from National Science Funding of China, Grant #22302815 from the Hong Kong Research Grant Council, and Grant FRG2/14-15/069 from Hong Kong Baptist University (the Hong Kong RGC grant HKBU); the NIH Grant R01-GM59507, the CTSA Grant UL1-RR024139, and the Department of Veterans Affairs (VA Cooperative Studies Program).

DOI

10.1214/15-AOS1421

ISSN (print)

00905364

Link to Publisher’s Edition

http://dx.doi.org/10.1214/15-AOS1421

Keywords

Asymptotic property, heritability, misspecified LMM, MMMA, random matrix theory, REML, variance components.

Additional Files

JA-5227-28989_publisher_suppl.pdf (91 kB)

Included in

Mathematics Commons

Share

COinS