Finding the power to reduce publication bias

• Published in: Statistics in medicine Vol. 36; no. 10; pp. 1580 - 1598
• Main Authors: Stanley, T. D., Doucouliagos, Hristos, Ioannidis, John P. A.
• Format: Journal Article
• Language: English
• Published: England Wiley Subscription Services, Inc 10.05.2017
• Subjects: Biostatistics; Computer Simulation; Humans; Least-Squares Analysis; Meta-Analysis as Topic; meta‐analysis; Models, Statistical; Odds Ratio; publication bias; Publication Bias - statistics & numerical data; random‐effects; Regression Analysis; statistical power; weighted least squares; meta-analysis; statistical power; publication bias; random-effects; weighted least squares
• ISSN: 0277-6715; 1097-0258
• DOI: 10.1002/sim.7228

The central purpose of this study is to document how a sharper focus upon statistical power may reduce the impact of selective reporting bias in meta‐analyses. We introduce the weighted average of the adequately powered (WAAP) as an alternative to the conventional random‐effects (RE) estimator. When the results of some of the studies have been selected to be positive and statistically significant (i.e. selective reporting), our simulations show that WAAP will have smaller bias than RE at no loss to its other statistical properties. When there is no selective reporting, the difference between RE's and WAAP's statistical properties is practically negligible. Nonetheless, when selective reporting is especially severe or heterogeneity is very large, notable bias can remain in all weighted averages. The main limitation of this approach is that the majority of meta‐analyses of medical research do not contain any studies with adequate power (i.e. >80%). For such areas of medical research, it remains important to document their low power, and, as we demonstrate, an alternative unrestricted weighted least squares weighted average can be used instead of WAAP. Copyright © 2017 John Wiley & Sons, Ltd.