Replication and contradiction of highly cited research papers inpsychiatry: 10-year follow-up

• Published in: British journal of psychiatry Vol. 207; no. 4; pp. 357 - 362
• Main Authors: Tajika Aran, Ogawa Yusuke, Takeshima Nozomi, Hayasaka Yu, Furukawa, Toshi A
• Format: Journal Article
• Language: English; Japanese
• Published: London Cambridge University Press 01.10.2015
• Subjects: Alzheimer's disease; Bias; Citations; Drug dosages; Epidemiology; Impact factors; Medicine; Psychiatry; Replication; Studies
• ISSN: 0007-1250; 1472-1465
• DOI: 10.1192/bjp.bp.113.143701

BackgroundContradictions and initial overestimates are not unusual among highlycited studies. However, this issue has not been researched inpsychiatry.AimsTo assess how highly cited studies in psychiatry are replicated bysubsequent studies.MethodWe selected highly cited studies claiming effective psychiatrictreatments in the years 2000 through 2002. For each of these studies wesearched for subsequent studies with a better-controlled design, or witha similar design but a larger sample.ResultsAmong 83 articles recommending effective interventions, 40 had not beensubject to any attempt at replication, 16 were contradicted, 11 werefound to have substantially smaller effects and only 16 were replicated.The standardised mean differences of the initial studies wereoverestimated by 132%. Studies with a total sample size of 100 or moretended to produce replicable results.ConclusionsCaution is needed when a study with a small sample size reports a largeeffect.