Department of Mathematics
We study behavior of the restricted maximum likelihood (REML) estimator under a misspecified linear mixed model (LMM) that has received much attention in recent genome-wide association studies. The asymptotic analysis establishes consistency of the REML estimator of the variance of the errors in the LMM, and convergence in probability of the REML estimator of the variance of the random effects in the LMM to a certain limit, which is equal to the true variance of the random effects multiplied by the limiting proportion of the nonzero random effects present in the LMM. The asymptotic results also establish convergence rate (in probability) of the REML estimators as well as a result regarding convergence of the asymptotic conditional variance of the REML estimator. The asymptotic results are fully supported by the results of empirical studies, which include extensive simulation studies that compare the performance of the REML estimator (under the misspecified LMM) with other existing methods, and real data applications (only one example is presented) that have important genetic implications.
Asymptotic property, heritability, misspecified LMM, MMMA, random matrix theory, REML, variance components.
Source Publication Title
Annals of Statistics
Institute of Mathematical Statistics
Supported in part by the NSF Grants DMS-08-09127, SES-1121794, and the NIH Grant R01-GM085205A1; the NIH Grant R01-GM59507;the NSF Grants DMS-11-06690 and DMS-1407530; Grant #61501389 from National Science Funding of China, Grant #22302815 from the Hong Kong Research Grant Council, and Grant FRG2/14-15/069 from Hong Kong Baptist University (the Hong Kong RGC grant HKBU); the NIH Grant R01-GM59507, the CTSA Grant UL1-RR024139, and the Department of Veterans Affairs (VA Cooperative Studies Program).
Link to Publisher's Edition
Jiang, J., Li, C., Debashis, P., Yang, C., & Zhao, H. (2016). On high-dimensional misspecified mixed model analysis in genome-wide association study. Annals of Statistics, 44 (5). https://doi.org/10.1214/15-AOS1421