A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings

• Published in: Consciousness and cognition Vol. 21; no. 1; pp. 422 - 430
• Main Authors: Maniscalco, Brian, Lau, Hakwan
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier 01.03.2012; Elsevier BV
• Subjects: Biological and medical sciences; Cognition; Cognition & reasoning; Cognition. Intelligence; Confidence; Fundamental and applied biological sciences. Psychology; Humans; Judgment; Metacognition; Miscellaneous; Psychological Theory; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Self image; Sensitivity and Specificity; Signal Detection, Psychological; Signal-To-Noise Ratio; Space Perception; Human; Signal detection theory; Sensitivity; Confidence rating; Confidence; Metacognition; Type 2 sensitivity; Cognition; Measurement method; Psychophysics; Signal detection
• ISSN: 1053-8100; 1090-2376; 1090-2376
• DOI: 10.1016/j.concog.2011.09.021

How should we measure metacognitive ("type 2") sensitivity, i.e. the efficacy with which observers' confidence ratings discriminate between their own correct and incorrect stimulus classifications? We argue that currently available methods are inadequate because they are influenced by factors such as response bias and type 1 sensitivity (i.e. ability to distinguish stimuli). Extending the signal detection theory (SDT) approach of Galvin, Podd, Drga, and Whitmore (2003), we propose a method of measuring type 2 sensitivity that is free from these confounds. We call our measure meta-d', which reflects how much information, in signal-to-noise units, is available for metacognition. Applying this novel method in a 2-interval forced choice visual task, we found that subjects' metacognitive sensitivity was close to, but significantly below, optimality. We discuss the theoretical implications of these findings, as well as related computational issues of the method. We also provide free Matlab code for implementing the analysis.