Empirical Evidence of Bias in the Design of Experimental Stroke Studies : A Metaepidemiologic Approach

• Published in: Stroke (1970) Vol. 39; no. 3; pp. 929 - 934
• Main Authors: CROSSLEY, Nicolas A, SENA, Emily, GOEHLER, Jos, HORN, Jannekke, VAN DER WORP, Bart, BATH, Philip M. W, MACLEOD, Malcolm, DIMAGL, Ulrich
• Format: Journal Article
• Language: English
• Published: Hagerstown, MD Lippincott Williams & Wilkins 01.03.2008
• Subjects: Animals; Bias; Biological and medical sciences; Biomedical Research - standards; Brain Ischemia - complications; Brain Ischemia - drug therapy; Cerebral Infarction - etiology; Cerebral Infarction - pathology; Comorbidity; Epidemiologic Methods; Headache. Facial pains. Syncopes. Epilepsia. Intracranial hypertension. Brain oedema. Cerebral palsy; Medical sciences; Meta-Analysis as Topic; Nervous system (semeiology, syndromes); Neurology; Neuroprotective Agents - therapeutic use; Random Allocation; Reproducibility of Results; Research - standards; Single-Blind Method; Treatment Outcome; Vascular diseases and vascular malformations of the nervous system; Vascular disease; Stroke; Nervous system diseases; Animal; Central nervous system disease; animal experimentation; meta-analysis; Cardiovascular disease; Experimental study; Cerebrovascular disease; cerebrovascular accident; Cerebral disorder
• ISSN: 0039-2499; 1524-4628
• DOI: 10.1161/STROKEAHA.107.498725

At least part of the failure in the transition from experimental to clinical studies in stroke has been attributed to the imprecision introduced by problems in the design of experimental stroke studies. Using a metaepidemiologic approach, we addressed the effect of randomization, blinding, and use of comorbid animals on the estimate of how effectively therapeutic interventions reduce infarct size. Electronic and manual searches were performed to identify meta-analyses that described interventions in experimental stroke. For each meta-analysis thus identified, a reanalysis was conducted to estimate the impact of various quality items on the estimate of efficacy, and these estimates were combined in a meta-meta-analysis to obtain a summary measure of the impact of the various design characteristics. Thirteen meta-analyses that described outcomes in 15,635 animals were included. Studies that included unblinded induction of ischemia reported effect sizes 13.1% (95% CI, 26.4% to 0.2%) greater than studies that included blinding, and studies that included healthy animals instead of animals with comorbidities overstated the effect size by 11.5% (95% CI, 21.2% to 1.8%). No significant effect was found for randomization, blinded outcome assessment, or high aggregate CAMARADES quality score. We provide empirical evidence of bias in the design of studies, with studies that included unblinded induction of ischemia or healthy animals overestimating the effectiveness of the intervention. This bias could account for the failure in the transition from bench to bedside of stroke therapies.