Robust Bayesian meta‐analysis: Model‐averaging across complementary publication bias adjustment methods

Publication bias is a ubiquitous threat to the validity of meta‐analysis and the accumulation of scientific evidence. In order to estimate and counteract the impact of publication bias, multiple methods have been developed; however, recent simulation studies have shown the methods' performance...

Full description

Saved in:
Bibliographic Details
Published inResearch synthesis methods Vol. 14; no. 1; pp. 99 - 116
Main Authors Bartoš, František, Maier, Maximilian, Wagenmakers, Eric‐Jan, Doucouliagos, Hristos, Stanley, T. D.
Format Journal Article
LanguageEnglish
Published England Wiley 01.01.2023
Wiley Subscription Services, Inc
John Wiley and Sons Inc
Subjects
Online AccessGet full text

Cover

Loading…
More Information
Summary:Publication bias is a ubiquitous threat to the validity of meta‐analysis and the accumulation of scientific evidence. In order to estimate and counteract the impact of publication bias, multiple methods have been developed; however, recent simulation studies have shown the methods' performance to depend on the true data generating process, and no method consistently outperforms the others across a wide range of conditions. Unfortunately, when different methods lead to contradicting conclusions, researchers can choose those methods that lead to a desired outcome. To avoid the condition‐dependent, all‐or‐none choice between competing methods and conflicting results, we extend robust Bayesian meta‐analysis and model‐average across two prominent approaches of adjusting for publication bias: (1) selection models of p‐values and (2) models adjusting for small‐study effects. The resulting model ensemble weights the estimates and the evidence for the absence/presence of the effect from the competing approaches with the support they receive from the data. Applications, simulations, and comparisons to preregistered, multi‐lab replications demonstrate the benefits of Bayesian model‐averaging of complementary publication bias adjustment methods.
Bibliography:Funding information
NWO, Grant/Award Number: 016.Vici.170.083
ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 23
Funding information NWO, Grant/Award Number: 016.Vici.170.083
ISSN:1759-2879
1759-2887
DOI:10.1002/jrsm.1594