Person-Fit as an Index of Inattentive Responding: A Comparison of Methods Using Polytomous Survey Data

• Published in: Applied psychological measurement Vol. 43; no. 5; pp. 374 - 387
• Main Authors: Beck, Mark F., Albano, Anthony D., Smith, Wendy M.
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.07.2019
• Subjects: nonparametric IRT; response biases; data quality; inattentive responding; person-fit statistics
• ISSN: 0146-6216; 1552-3497
• DOI: 10.1177/0146621618798666

Self-report measures are vulnerable to response biases that can degrade the accuracy of conclusions drawn from results. In low-stakes measures, inattentive or careless responding can be especially problematic. A variety of a priori and post hoc methods exist for detecting these aberrant response patterns. Previous research indicates that nonparametric person-fit statistics tend to be the most accurate post hoc method for detecting inattentive responding on measures with dichotomous outcomes. This study investigated the accuracy and impact on model fit of parametric and nonparametric person-fit statistics in detecting inattentive responding with polytomous response scales. Receiver operating curve (ROC) analysis was used to determine the accuracy of each detection metric, and confirmatory factor analysis (CFA) fit indices were used to examine the impact of using person-fit statistics to identify inattentive respondents. ROC analysis showed the nonparametric HT statistic offered the most area under the curve when predicting a proxy for inattentive responding. The CFA fit indices showed the impact of using the person-fit statistics largely depends on the purpose (and cutoff) for using the person-fit statistics. Implications for using person-fit statistics to identify inattentive responders are discussed further.