Outliers detection in assessment tests’ quality evaluation through the blended use of functional data analysis and item response theory

• Published in: Annals of operations research Vol. 342; no. 3; pp. 1547 - 1562
• Main Authors: Maturo, Fabrizio, Fortuna, Francesca, Di Battista, Tonio
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.11.2024; Springer Nature B.V
• Subjects: Business and Management; Combinatorics; Data analysis; Functionals; Item response theory; Operations Research/Decision Theory; Original Research; Outliers (statistics); Quality assessment; Questionnaires; Theory of Computation; ICC; FDA; IRT; Questionnaire quality; Log odds-ratio; Functional outlier detection
• ISSN: 0254-5330; 1572-9338
• DOI: 10.1007/s10479-022-05099-z

The quality of assessment tests plays a fundamental role in decision-making problems in various fields such as education, psychology, and behavioural medicine. The first phase in the questionnaires’ validation process is outliers’ recognition. The latter can be identified at different levels, such as subject responses, individuals, and items. This paper focuses on item outliers and proposes a blended use of functional data analysis and item response theory for identifying outliers in assessment tests. The basic idea is that item characteristics curves derived from test responses can be treated as functions, and functional tools can be exploited to discover anomalies in item behaviour. For this purpose, this research suggests a multi-step strategy to catch magnitude and shape outliers employing a suitable transformation of item characteristics curves and their first derivatives. A simulation study emphasises the effectiveness of the proposed technique and exhibits exciting results in discovering outliers that classical functional methods do not detect. Moreover, the applicability of the method is shown with a real dataset. The final aim is to offer a methodology for improving the questionnaires’ quality.