Automated gaze-based mind wandering detection during computerized learning in classrooms

• Published in: User modeling and user-adapted interaction Vol. 29; no. 4; pp. 821 - 867
• Main Authors: Hutt, Stephen, Krasich, Kristina, Mills, Caitlin, Bosch, Nigel, White, Shelby, Brockmole, James R., D’Mello, Sidney K.
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.09.2019; Springer Nature B.V
• Subjects: Accuracy; Automation; Classrooms; Commercial off-the-shelf technology; Computer Science; Detectors; Eye movements; Feasibility studies; Learning; Management of Computing and Information Systems; Missing data; Multimedia Information Systems; Students; Tracking; User Interfaces and Human Computer Interaction; Attention-aware learning; Cyberlearning; Mind wandering; Eye-gaze; Intelligent tutoring systems
• ISSN: 0924-1868; 1573-1391
• DOI: 10.1007/s11257-019-09228-5

We investigate the use of commercial off-the-shelf (COTS) eye-trackers to automatically detect mind wandering—a phenomenon involving a shift in attention from task-related to task-unrelated thoughts—during computerized learning. Study 1 ( N  = 135 high-school students) tested the feasibility of COTS eye tracking while students learn biology with an intelligent tutoring system called GuruTutor in their classroom. We could successfully track eye gaze in 75% (both eyes tracked) and 95% (one eye tracked) of the cases for 85% of the sessions where gaze was successfully recorded. In Study 2, we used this data to build automated student-independent detectors of mind wandering, obtaining accuracies (mind wandering F 1  = 0.59) substantially better than chance (F 1  = 0.24). Study 3 investigated context-generalizability of mind wandering detectors, finding that models trained on data collected in a controlled laboratory more successfully generalized to the classroom than the reverse. Study 4 investigated gaze- and video- based mind wandering detection, finding that gaze-based detection was superior and multimodal detection yielded an improvement in limited circumstances. We tested live mind wandering detection on a new sample of 39 students in Study 5 and found that detection accuracy (mind wandering F 1  = 0.40) was considerably above chance (F1 = 0.24), albeit lower than offline detection accuracy from Study 1 (F 1  = 0.59), a finding attributable to handling of missing data. We discuss our next steps towards developing gaze-based attention-aware learning technologies to increase engagement and learning by combating mind wandering in classroom contexts.