Multiple Testing of Composite Null Hypotheses in Heteroscedastic Models

• Published in: Journal of the American Statistical Association Vol. 107; no. 498; pp. 673 - 687
• Main Authors: Sun, Wenguang, McLain, Alexander C
• Format: Journal Article
• Language: English
• Published: Alexandria Taylor & Francis Group 01.06.2012; Taylor & Francis Ltd
• Subjects: Ambivalence; Applied statistics; Compound decision theory; Consistent estimators; data analysis; Deconvoluting kernel density estimator; Density estimation; Deviation; Estimation; Estimators; False important discovery rate; heteroskedasticity; Hypotheses; Hypothesis; Indifference region; Large-scale simultaneous inference; Mathematical procedures; Null hypothesis; Oracles; P values; Regression analysis; Size; Standardization; Statistical methods; Statistical theories; Statistics; Testing; Theory and Methods; Variance analysis
• DOI: 10.1080/01621459.2012.664505
• ISSN: 1537-274X; 0162-1459

In large-scale studies, the true effect sizes often range continuously from zero to small to large, and are observed with heteroscedastic errors. In practical situations where the failure to reject small deviations from the null is inconsequential, specifying an indifference region (or forming composite null hypotheses) can greatly reduce the number of unimportant discoveries in multiple testing. The heteroscedasticity issue poses new challenges for multiple testing with composite nulls. In particular, the conventional framework in multiple testing, which involves rescaling or standardization, is likely to distort the scientific question. We propose the concept of a composite null distribution for heteroscedastic models and develop an optimal testing procedure that minimizes the false nondiscovery rate, subject to a constraint on the false discovery rate. The proposed approach is different from conventional methods in that the effect size, statistical significance, and multiplicity issues are addressed integrally. The external information of heteroscedastic errors is incorporated for optimal simultaneous inference. The new features and advantages of our approach are demonstrated using both simulated and real data. The numerical studies demonstrate that our new procedure enjoys superior performance with greater accuracy and better interpretability of results.