Learning analytics for student modeling in virtual reality training systems: Lineworkers case

• Published in: Computers and education Vol. 151; p. 103871
• Main Authors: Santamaría-Bonfil, Guillermo, Ibáñez, María Blanca, Pérez-Ramírez, Miguel, Arroyo-Figueroa, Gustavo, Martínez-Álvarez, Francisco
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.07.2020
• Subjects: Exploratory data analysis; Feature importance analysis; Learning analytics; Performance prediction; Virtual reality; Performance prediction; Learning analytics; Feature importance analysis; Exploratory data analysis; Virtual reality
• ISSN: 0360-1315; 1873-782X
• DOI: 10.1016/j.compedu.2020.103871

Live-line maintenance is a high risk activity. Hence, lineworkers require effective and safe training. Virtual Reality Training Systems (VRTS) provide an affordable and safe alternative for training in such high risk environments. However, their effectiveness relies mainly on having meaningful activities for supporting learning and on their ability to detect untrained students. This study builds a student model based on Learning Analytics (LA), using data collected from 1399 students that used a VRTS for the maintenance training of lineworkers in 329 courses carried out from 2008 to 2016. By employing several classifiers, the model allows discriminating between trained and untrained students in different maneuvers using three minimum evaluation proficiency scores. Using the best classifier, a Feature Importance Analysis is carried out to understand the impact of the variables regarding the trainees’ final performances. The model also involves the exploration of the trainees’ trace data through a visualization tool to pose non-observable behavioral variables related to displayed errors. The results show that the model can discriminate between trained and untrained students, the Random Forest algorithm standing out. The feature importance analysis revealed that the most relevant features regarding the trainees’ final performance were profile and course variables along with specific maneuver steps. Finally, using the visual tool, and with human expert aid, several error patterns in trace data associated with misconceptions and confusion were identified. In the light of these, LA enables disassembling the data jigsaw quandary from VRTS to enhance the human-in-the-loop evaluation. •Learning analytics is used to examine training in high-risk virtual training system.•Trainees final performance is predicted using several classifiers.•Increasing evaluation strictness improves final performance prediction.•Feature analysis shows variables importance and contribution to final performance.•A visualization tool is used to unveil non-observable behavioral variables.