Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): Random permutations and cluster size control

• Published in: NeuroImage (Orlando, Fla.) Vol. 65; pp. 69 - 82
• Main Authors: Stelzer, Johannes, Chen, Yi, Turner, Robert
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier Inc 15.01.2013; Elsevier; Elsevier Limited
• Subjects: Binomial distribution; Biological and medical sciences; Brain Mapping - methods; Classification; Cluster size control; fMRI; Fundamental and applied biological sciences. Psychology; Humans; Hypotheses; Image Processing, Computer-Assisted - methods; Magnetic Resonance Imaging - methods; Methods; Models, Neurological; Models, Theoretical; Multiple testing; MVPA; Probability distribution; Second level analysis; Sensitivity and Specificity; Software; Statistics; Vertebrates: nervous system and sense organs; Second level analysis; Cluster size control; fMRI; Multiple testing; Statistics; MVPA; Nuclear magnetic resonance imaging; Functional imaging
• ISSN: 1053-8119; 1095-9572; 1095-9572
• DOI: 10.1016/j.neuroimage.2012.09.063

An ever-increasing number of functional magnetic resonance imaging (fMRI) studies are now using information-based multi-voxel pattern analysis (MVPA) techniques to decode mental states. In doing so, they achieve a significantly greater sensitivity compared to when they use univariate frameworks. However, the new brain-decoding methods have also posed new challenges for analysis and statistical inference on the group level. We discuss why the usual procedure of performing t-tests on accuracy maps across subjects in order to produce a group statistic is inappropriate. We propose a solution to this problem for local MVPA approaches, which achieves higher sensitivity than other procedures. Our method uses random permutation tests on the single-subject level, and then combines the results on the group level with a bootstrap method. To preserve the spatial dependency induced by local MVPA methods, we generate a random permutation set and keep it fixed across all locations. This enables us to later apply a cluster size control for the multiple testing problem. More specifically, we explicitly compute the distribution of cluster sizes and use this to determine the p-values for each cluster. Using a volumetric searchlight decoding procedure, we demonstrate the validity and sensitivity of our approach using both simulated and real fMRI data sets. In comparison to the standard t-test procedure implemented in SPM8, our results showed a higher sensitivity. We discuss the theoretical applicability and the practical advantages of our approach, and outline its generalization to other local MVPA methods, such as surface decoding techniques. ► New group statistics for classification-based brain decoding ► Based on permutation tests and bootstrap methods ► Higher statistical sensitivity than t-test frameworks ► Higher spatial specificity than t-test frameworks