No evidence of important difference in summary treatment effects between COVID-19 preprints and peer-reviewed publications: a meta-epidemiological study

• Published in: Journal of clinical epidemiology Vol. 162; pp. 90 - 97
• Main Authors: Davidson, Mauricia, Evrenoglou, Theodoros, Graña, Carolina, Chaimani, Anna, Boutron, Isabelle
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.10.2023; Elsevier Limited; Elsevier
• Subjects: Clinical trials; Coronaviruses; COVID-19; COVID-19 - epidemiology; Drug therapy; Epidemiologic Studies; Epidemiology; Estimates; Humans; Life Sciences; Meta-analysis; Meta-epidemiology; Pandemics; Peer Review; Preprint; Randomized controlled trial; Sample Size; Statistical analysis; COVID-19; Meta-epidemiology; Peer-review; Randomized controlled trial; Preprint; Meta-analysis; Meta -analysis
• ISSN: 0895-4356; 1878-5921; 1878-5921
• DOI: 10.1016/j.jclinepi.2023.08.011

Preprints became a major source of research communication during the COVID-19 pandemic. We aimed to evaluate whether summary treatment effect estimates differ between preprint and peer-reviewed journal trials. A meta-epidemiological study. Data were derived from the COVID-NMA living systematic review (covid-nma.com) up to July 20, 2022. We identified all meta-analyses evaluating pharmacological treatments vs. standard of care or placebo for patients with COVID-19 that included at least one preprint and one peer-reviewed journal article. Difference in effect estimates between preprint and peer-reviewed journal trials were estimated by the ratio of odds ratio (ROR); ROR <1 indicated larger effects in preprint trials. Thirty-seven meta-analyses including 114 trials (44 preprints and 70 peer-reviewed publications) were selected. The median number of randomized controlled trials (RCTs) per meta-analysis was 2 (interquartile range [IQR], 2–4; maximum, 11), median sample size of RCTs was 199 (IQR, 99–478). Overall, there was no statistically significant difference in summary effect estimates between preprint and peer-reviewed journal trials (ROR, 0.88; 95% CI, 0.71–1.09; I2 = 17.8%; τ2 = 0.06). We did not find an important difference between summary treatment effects of preprints and summary treatment effects of peer-reviewed publications. Systematic reviewers and guideline developers should assess preprint inclusion individually, accounting for risk of bias and completeness of reporting.