re-evaluation of random-effects meta-analysis

• Published in: Journal of the Royal Statistical Society. Series A, Statistics in society Vol. 172; no. 1; pp. 137 - 159
• Main Authors: Higgins, Julian P. T., Thompson, Simon G., Spiegelhalter, David J.
• Format: Journal Article
• Language: English
• Published: Oxford, UK Oxford, UK : Blackwell Publishing Ltd 01.01.2009; Blackwell Publishing Ltd; Blackwell Publishing; Blackwell; Royal Statistical Society; Oxford University Press
• Series: Journal of the Royal Statistical Society Series A
• Subjects: Applications; Bayesian analysis; Bayesian method; Confidence interval; Degrees of freedom; Distribution theory; Eating disorders; Exact sciences and technology; Forecasts; Gaussian distributions; General topics; Hypothesis; Hypothesis testing; Inference; Mathematical intervals; Mathematics; Meta analysis; Methodology; Modeling; Normal distribution; Null hypothesis; Original; Prediction; Predictions; Probability and statistics; Probability theory and stochastic processes; Random-effects models; Sciences and techniques of general use; Standard error; Statistical methods; Statistical variance; Statistics; Stochastic processes; Systematic reviews; Bayes estimation; Random-effects models; Prediction; Gaussian distribution; Prediction theory; Non parametric estimation; Statistical estimation; Multivariate analysis; Random distribution; Stochastic process; Mean estimation; Prediction interval; Meta-analysis; Statistical method; Hypothesis test; Heterogeneity; Statistical test; Distribution function; Systematic reviews; Random effect; Filtering theory; Application
• ISSN: 0964-1998; 1467-985X
• DOI: 10.1111/j.1467-985X.2008.00552.x

Meta-analysis in the presence of unexplained heterogeneity is frequently undertaken by using a random-effects model, in which the effects underlying different studies are assumed to be drawn from a normal distribution. Here we discuss the justification and interpretation of such models, by addressing in turn the aims of estimation, prediction and hypothesis testing. A particular issue that we consider is the distinction between inference on the mean of the random-effects distribution and inference on the whole distribution. We suggest that random-effects meta-analyses as currently conducted often fail to provide the key results, and we investigate the extent to which distribution-free, classical and Bayesian approaches can provide satisfactory methods. We conclude that the Bayesian approach has the advantage of naturally allowing for full uncertainty, especially for prediction. However, it is not without problems, including computational intensity and sensitivity to a priori judgements. We propose a simple prediction interval for classical meta-analysis and offer extensions to standard practice of Bayesian meta-analysis, making use of an example of studies of 'set shifting' ability in people with eating disorders.