The Importance of Demographically Adjusted Cutoffs: Age and Education Bias in Raw Score Cutoffs Within the Trail Making Test

• Published in: Psychological injury and law Vol. 12; no. 2; pp. 170 - 182
• Main Authors: Abeare, Christopher, Sabelli, Alana, Taylor, Brian, Holcomb, Matthew, Dumitrescu, Claudiu, Kirsch, Ned, Erdodi, Laszlo
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.06.2019; Springer Nature B.V
• Subjects: Accuracy; Achievement tests; Age; Behavioral Science and Psychology; Bias; Brain research; Classification; Clinical Psychology; Demography; Education; False positive results; Law and Psychology; Memory; Neuropsychological assessment; Neuropsychology; Older people; Outpatients; Psychology; Traumatic brain injury; Validity; Performance validity; Traumatic brain injury; Erdodi Index; Demographic correction; Trail making test
• ISSN: 1938-971X; 1938-9728
• DOI: 10.1007/s12207-019-09353-x

This study was designed to develop validity cutoffs by utilizing demographically adjusted T-scores on the trail making test (TMT), with the goal of eliminating potential age and education-related biases associated with the use of raw score cutoffs. Failure to correct for the effect of age and education on TMT performance may lead to increased false positive errors for older adults and examinees with lower levels of education. Data were collected from an archival sample of 100 adult outpatients ( M Age  = 38.8, 56% male; M Ed  = 13.7) who were clinically referred for neuropsychological assessment at an academic medical center in the Midwestern USA after sustaining a traumatic brain injury (TBI). Performance validity was psychometrically determined using the Word Memory Test and two multivariate validity composites based on five embedded performance validity indicators. Cutoffs on the demographically corrected TMT T-scores had generally superior classification accuracy compared to the raw score cutoffs reported in the literature. As expected, the T-scores also eliminated age and education bias that was observed in the raw score cutoffs. Both T-score and raw score cutoffs were orthogonal to injury severity. Multivariate models of T-score based cutoff failed to improve classification accuracy over univariate T-score cutoffs. The present findings provide support for the use of demographically adjusted validity cutoffs within the TMT. They produced superior classification to raw score-based cutoffs, in addition to eliminating the bias against older adults and examinees with lower levels of education.