From “below chance” to “a single error is one too many”: Evaluating various thresholds for invalid performance on two forced choice recognition tests

• Published in: Behavioral sciences & the law Vol. 41; no. 5; pp. 445 - 462
• Main Author: Erdodi, Laszlo A.
• Format: Journal Article
• Language: English
• Published: United States Wiley Subscription Services, Inc 01.09.2023
• Subjects: Acknowledgment; Binomial distribution; Dementia; Humans; Malingering; Memory and Learning Tests; Recognition; Recognition, Psychology; Reproducibility of Results; Tests; Thresholds; chance level performance; performance validity; FCRCVLT-II; TOMM-2; binomial distribution
• ISSN: 0735-3936; 1099-0798; 1099-0798
• DOI: 10.1002/bsl.2609

This study was designed to empirically evaluate the classification accuracy of various definitions of invalid performance in two forced‐choice recognition performance validity tests (PVTs; FCR CVLT‐II and Test of Memory Malingering [TOMM‐2]). The proportion of at and below chance level responding defined by the binomial theory and making any errors was computed across two mixed clinical samples from the United States and Canada ( N  = 470) and two sets of criterion PVTs. There was virtually no overlap between the binomial and empirical distributions. Over 95% of patients who passed all PVTs obtained a perfect score. At chance level responding was limited to patients who failed ≥2 PVTs (91% of them failed 3 PVTs). No one scored below chance level on FCR CVLT‐II or TOMM‐2. All 40 patients with dementia scored above chance. Although at or below chance level performance provides very strong evidence of non‐credible responding, scores above chance level have no negative predictive value. Even at chance level scores on PVTs provide compelling evidence for non‐credible presentation. A single error on the FCR CVLT‐II or TOMM‐2 is highly specific (0.95) to psychometrically defined invalid performance. Defining non‐credible responding as below chance level scores is an unnecessarily restrictive threshold that gives most examinees with invalid profiles a Pass .