Test-retest and between-site reliability in a multicenter fMRI study

• Published in: Human brain mapping Vol. 29; no. 8; pp. 958 - 972
• Main Authors: Friedman, Lee, Stern, Hal, Brown, Gregory G., Mathalon, Daniel H., Turner, Jessica, Glover, Gary H., Gollub, Randy L., Lauriello, John, Lim, Kelvin O., Cannon, Tyrone, Greve, Douglas N., Bockholt, Henry Jeremy, Belger, Aysenil, Mueller, Bryon, Doty, Michael J., He, Jianchun, Wells, William, Smyth, Padhraic, Pieper, Steve, Kim, Seyoung, Kubicki, Marek, Vangel, Mark, Potkin, Steven G.
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc., A Wiley Company 01.08.2008; Wiley-Liss
• Subjects: Adult; Biological and medical sciences; Electrodiagnosis. Electric activity recording; FMRI; Humans; intraclass correlation coefficient; Investigative techniques, diagnostic techniques (general aspects); Magnetic Resonance Imaging - standards; Male; Medical sciences; multicenter; Multicenter Studies as Topic - methods; Multicenter Studies as Topic - standards; Nervous system; Radiodiagnosis. Nmr imagery. Nmr spectrometry; reproducibility; Research Design - standards; test-retest; Correlation coefficient; Nervous system diseases; Radiodiagnosis; Reproducibility; Multicenter study; multicenter; test-retest; intraclass correlation coefficient; Test reliability; Nuclear magnetic resonance imaging; FMRI
• ISSN: 1065-9471; 1097-0193; 1097-0193
• DOI: 10.1002/hbm.20440

In the present report, estimates of test–retest and between‐site reliability of fMRI assessments were produced in the context of a multicenter fMRI reliability study (FBIRN Phase 1, www.nbirn.net). Five subjects were scanned on 10 MRI scanners on two occasions. The fMRI task was a simple block design sensorimotor task. The impulse response functions to the stimulation block were derived using an FIR‐deconvolution analysis with FMRISTAT. Six functionally‐derived ROIs covering the visual, auditory and motor cortices, created from a prior analysis, were used. Two dependent variables were compared: percent signal change and contrast‐to‐noise‐ratio. Reliability was assessed with intraclass correlation coefficients derived from a variance components analysis. Test–retest reliability was high, but initially, between‐site reliability was low, indicating a strong contribution from site and site‐by‐subject variance. However, a number of factors that can markedly improve between‐site reliability were uncovered, including increasing the size of the ROIs, adjusting for smoothness differences, and inclusion of additional runs. By employing multiple steps, between‐site reliability for 3T scanners was increased by 123%. Dropping one site at a time and assessing reliability can be a useful method of assessing the sensitivity of the results to particular sites. These findings should provide guidance toothers on the best practices for future multicenter studies. Hum Brain Mapp, 2008. © 2007 Wiley‐Liss, Inc.