Estimating Multilevel Logistic Regression Models When the Number of Clusters is Low: A Comparison of Different Statistical Software Procedures

• Published in: The International Journal of Biostatistics Vol. 6; no. 1; pp. 16 - 35
• Main Author: Austin, Peter C
• Format: Journal Article
• Language: English
• Published: Germany bepress 22.04.2010; De Gruyter; Berkeley Electronic Press
• Subjects: Bayes Theorem; Bayesian analysis; BUGS; Cluster Analysis; Computation; Data Interpretation, Statistical; General Biostatistics; generalized linear mixed models; Health Services Research; hierarchical models; Humans; Linear Models; Logistic Models; mixed effects model; Models, Statistical; Monte Carlo Method; Monte Carlo simulations; multilevel models; random effects model; SAS; Software; Stata; statistical software; BUGS; R; random effects model; Monte Carlo simulations; SAS; hierarchical models; Stata; generalized linear mixed models; multilevel models; statistical software; Bayesian analysis; mixed effects model
• DOI: 10.2202/1557-4679.1195
• ISSN: 1557-4679

Abstract Multilevel logistic regression models are increasingly being used to analyze clustered data in medical, public health, epidemiological, and educational research. Procedures for estimating the parameters of such models are available in many statistical software packages. There is currently little evidence on the minimum number of clusters necessary to reliably fit multilevel regression models. We conducted a Monte Carlo study to compare the performance of different statistical software procedures for estimating multilevel logistic regression models when the number of clusters was low. We examined procedures available in BUGS, HLM, R, SAS, and Stata. We found that there were qualitative differences in the performance of different software procedures for estimating multilevel logistic models when the number of clusters was low. Among the likelihood-based procedures, estimation methods based on adaptive Gauss-Hermite approximations to the likelihood (glmer in R and xtlogit in Stata) or adaptive Gaussian quadrature (Proc NLMIXED in SAS) tended to have superior performance for estimating variance components when the number of clusters was small, compared to software procedures based on penalized quasi-likelihood. However, only Bayesian estimation with BUGS allowed for accurate estimation of variance components when there were fewer than 10 clusters. For all statistical software procedures, estimation of variance components tended to be poor when there were only five subjects per cluster, regardless of the number of clusters. Recommended Citation Austin, Peter C. (2010) "Estimating Multilevel Logistic Regression Models When the Number of Clusters is Low: A Comparison of Different Statistical Software Procedures," The International Journal of Biostatistics: Vol. 6 : Iss. 1, Article 16. DOI: 10.2202/1557-4679.1195 Available at: http://www.bepress.com/ijb/vol6/iss1/16