Data quality over data quantity in computational cognitive neuroscience

• Published in: NeuroImage (Orlando, Fla.) Vol. 172; pp. 775 - 785
• Main Authors: Kolossa, Antonio, Kopp, Bruno
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 15.05.2018; Elsevier Limited
• Subjects: Bayesian analysis; Behavioral sciences; Cognitive ability; Cognitive Neuroscience - methods; Cognitive Neuroscience - standards; Computational modeling; Data Accuracy; Experimental psychology; Functional brain imaging; Humans; Hypotheses; Hypothesis testing; Models, Neurological; Nervous system; Neurosciences; Quantitative psychology; Reliability; Replicability; Sampling; Sampling error; Signal-to-noise ratio; Statistics; Signal-to-noise ratio; Functional brain imaging; Computational modeling; Reliability; Replicability
• ISSN: 1053-8119; 1095-9572; 1095-9572
• DOI: 10.1016/j.neuroimage.2018.01.005

We analyzed factors that may hamper the advancement of computational cognitive neuroscience (CCN). These factors include a particular statistical mindset, which paves the way for the dominance of statistical power theory and a preoccupation with statistical replicability in the behavioral and neural sciences. Exclusive statistical concerns about sampling error occur at the cost of an inadequate representation of the problem of measurement error. We contrasted the manipulation of data quantity (sampling error, by varying the number of subjects) against the manipulation of data quality (measurement error, by varying the number of data per subject) in a simulated Bayesian model identifiability study. The results were clear-cut in showing that - across all levels of signal-to-noise ratios - varying the number of subjects was completely inconsequential, whereas the number of data per subject exerted massive effects on model identifiability. These results emphasize data quality over data quantity, and they call for the integration of statistics and measurement theory. •We define computational cognitive neuroscience within behavioral sciences.•Sampling error and data quantity are well-recognized in behavioral sciences.•We demonstrate the importance of measurement error and data quality.•We argue that statistics and psychometrics need merging.•We discuss problems related to the heterogeneity of populations.