Temporal analysis of multimodal data to predict collaborative learning outcomes

• Published in: British journal of educational technology Vol. 51; no. 5; pp. 1527 - 1547
• Main Authors: Olsen, Jennifer K., Sharma, Kshitij, Rummel, Nikol, Aleven, Vincent
• Format: Journal Article
• Language: English
• Published: Coventry Wiley 01.09.2020; Blackwell Publishing Ltd
• Subjects: Achievement Gains; Audio data; Collaborative learning; Cooperative Learning; Data analysis; Data Collection; Data transmission; Data Use; Educational Research; Eye Movements; Intelligent Tutoring Systems; Learning; Learning Analytics; Learning Modalities; Learning Processes; multimodal; Prediction
• ISSN: 0007-1013; 1467-8535
• DOI: 10.1111/bjet.12982

The analysis of multiple data streams is a long‐standing practice within educational research. Both multimodal data analysis and temporal analysis have been applied successfully, but in the area of collaborative learning, very few studies have investigated specific advantages of multiple modalities versus a single modality, especially combined with temporal analysis. In this paper, we investigate how both the use of multimodal data and moving from averages and counts to temporal aspects in a collaborative setting provides a better prediction of learning gains. To address these questions, we analyze multimodal data collected from 25 9–11‐year‐old dyads using a fractions intelligent tutoring system. Assessing the relation of dual gaze, tutor log, audio and dialog data to students' learning gains, we find that a combination of modalities, especially those at a smaller time scale, such as gaze and audio, provides a more accurate prediction of learning gains than models with a single modality. Our work contributes to the understanding of how analyzing multimodal data in temporal manner provides additional information around the collaborative learning process.