Multicraft: A Multimodal Interface for Supporting and Studying Learning in Minecraft

• Published in: HCI in Games: Serious and Immersive Games Vol. 12790; pp. 113 - 131
• Main Authors: Worsley, Marcelo, Mendoza Tudares, Kevin, Mwiti, Timothy, Zhen, Mitchell, Jiang, Marc
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Constructionism; Data mining; Games; Spatial reasoning

In this paper, we present work on bringing multimodal interaction to Minecraft. The platform, Multicraft, incorporates speech-based input, eye tracking, and natural language understanding to facilitate more equitable gameplay in Minecraft. We tested the platform with elementary, middle school studen...