Modeling NAEP Test-Taking Behavior Using Educational Process Analysis

• Published in: Journal of educational data mining Vol. 13; no. 2; pp. 16 - 54
• Main Authors: Patel, Nirmal, Sharma, Aditya, Shah, Tirth, Lomas, Derek
• Format: Journal Article
• Language: English
• Published: International Educational Data Mining 2021
• Subjects: Competition; Computer Assisted Testing; Evaluation Methods; Item Analysis; Learning Analytics; Markov Processes; Mathematics Tests; National Competency Tests; Prediction; Student Behavior; Test Items; Testing
• DOI: 10.5281/zenodo.5275312
• ISSN: 2157-2100

Process Analysis is an emerging approach to discover meaningful knowledge from temporal educational data. The study presented in this paper shows how we used Process Analysis methods on the National Assessment of Educational Progress (NAEP) test data for modeling and predicting student test-taking behavior. Our process-oriented data exploration gave us insightful findings of how students were interacting with the digital assessment system over time. To discover what processes students were following during the NAEP Digital Assessment, we first developed an innovative set of research questions. Then, we used Process Analysis methods to answer these questions and created a set of features that described student behavior over time. These features were used to create an ensemble model that aimed to accurately predict the digital test-taking efficiency of the students taking NAEP. Our model emerged as one of the most successful models in the 2019 NAEP Data Mining Competition, scoring second place out of 89 teams.