The relation between statistical power and inference in fMRI

• Published in: PloS one Vol. 12; no. 11; p. e0184923
• Main Authors: Cremers, Henk R., Wager, Tor D., Yarkoni, Tal
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 20.11.2017; Public Library of Science (PLoS)
• Subjects: Biology and Life Sciences; Brain; Brain - diagnostic imaging; Brain mapping; Brain research; Cognition & reasoning; Computer simulation; Connectome - statistics & numerical data; Consortia; Correlation; Dependent variables; Empirical analysis; Experiments; Extreme values; Functional magnetic resonance imaging; Group dynamics; Humans; Hypotheses; Magnetic resonance imaging; Magnetic Resonance Imaging - statistics & numerical data; Medical imaging; Medicine and Health Sciences; Meta-analysis; Methods; Neuroimaging; Neuroimaging - statistics & numerical data; Neurosciences; Physical Sciences; Research and Analysis Methods; Social Sciences; Statistical inference; Statistical methods; Statistical power; Statistics; Studies; Theory of Mind; Trends; United States
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0184923

Statistically underpowered studies can result in experimental failure even when all other experimental considerations have been addressed impeccably. In fMRI the combination of a large number of dependent variables, a relatively small number of observations (subjects), and a need to correct for multiple comparisons can decrease statistical power dramatically. This problem has been clearly addressed yet remains controversial-especially in regards to the expected effect sizes in fMRI, and especially for between-subjects effects such as group comparisons and brain-behavior correlations. We aimed to clarify the power problem by considering and contrasting two simulated scenarios of such possible brain-behavior correlations: weak diffuse effects and strong localized effects. Sampling from these scenarios shows that, particularly in the weak diffuse scenario, common sample sizes (n = 20-30) display extremely low statistical power, poorly represent the actual effects in the full sample, and show large variation on subsequent replications. Empirical data from the Human Connectome Project resembles the weak diffuse scenario much more than the localized strong scenario, which underscores the extent of the power problem for many studies. Possible solutions to the power problem include increasing the sample size, using less stringent thresholds, or focusing on a region-of-interest. However, these approaches are not always feasible and some have major drawbacks. The most prominent solutions that may help address the power problem include model-based (multivariate) prediction methods and meta-analyses with related synthesis-oriented approaches.