The effect of number of clusters and cluster size on statistical power and Type I error rates when testing random effects variance components in multilevel linear and logistic regression models

• Published in: Journal of statistical computation and simulation Vol. 88; no. 16; pp. 3151 - 3163
• Main Authors: Austin, Peter C., Leckie, George
• Format: Journal Article
• Language: English
• Published: Abingdon Taylor & Francis 02.11.2018; Taylor & Francis Ltd
• Subjects: Clusters; Computer simulation; Economic models; hierarchical model; Likelihood ratio; multilevel analysis; multilevel model; Null hypothesis; Regression analysis; Regression models; Statistical analysis; Statistical power; Variance; variance components
• DOI: 10.1080/00949655.2018.1504945
• ISSN: 0094-9655

When using multilevel regression models that incorporate cluster-specific random effects, the Wald and the likelihood ratio (LR) tests are used for testing the null hypothesis that the variance of the random effects distribution is equal to zero. We conducted a series of Monte Carlo simulations to examine the effect of the number of clusters and the number of subjects per cluster on the statistical power to detect a non-null random effects variance and to compare the empirical type I error rates of the Wald and LR tests. Statistical power increased with increasing number of clusters and number of subjects per cluster. Statistical power was greater for the LR test than for the Wald test. These results applied to both the linear and logistic regressions, but were more pronounced for the latter. The use of the LR test is preferable to the use of the Wald test.