Process Mining Combined with Expert Feature Engineering to Predict Efficient Use of Time on High-Stakes Assessments

• Published in: Journal of educational data mining Vol. 13; no. 2; pp. 1 - 15
• Main Author: Levin, Nathan A
• Format: Journal Article
• Language: English
• Published: International Educational Data Mining 2021
• Subjects: Classification; Competition; Computation; Computer Assisted Testing; Data Collection; Efficiency; Engineering; Grade 8; High Stakes Tests; Learning Analytics; Mathematics; Middle School Students; Prediction; Scores; Time Factors (Learning)
• ISSN: 2157-2100; 2157-2100
• DOI: 10.5281/zenodo.5275310

The Big Data for Education Spoke of the NSF Northeast Big Data Innovation Hub and ETS co-sponsored an educational data mining competition in which contestants were asked to predict efficient time use on the NAEP 8th grade mathematics computer-based assessment, based on the log file of a student's actions on a prior portion of the assessment. In this work, a combined approach of process mining and expert feature engineering was used to build a large set of features that were then trained with an Extreme Gradient Boosting machine learning model to classify students based on whether they would use their time efficiently. Predictions were evaluated throughout the competition on half of a hidden data set and then the final results were based on the second half of the hidden data set. The approach used here earned the top score in the competition. The work presented elaborates on the combined technique for analyzing computer-based assessment log-file data with the hope that this approach will offer valuable insights for future predictive model building in educational data mining.