Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher–AI Complementarity

• Published in: Journal of Learning Analytics Vol. 6; no. 2; pp. 27 - 52
• Main Authors: Holstein, Kenneth, McLaren, Bruce M., Aleven, Vincent
• Format: Journal Article
• Language: English
• Published: Society for Learning Analytics Research 01.01.2019
• Subjects: Artificial Intelligence; Design; Elementary Secondary Education; Learning Analytics; Middle School Teachers; Stakeholders; Teacher Participation; Technology Uses in Education
• ISSN: 1929-7750; 1929-7750
• DOI: 10.18608/jla.2019.62.3

Involving stakeholders throughout the creation of new educational technologies can help ensure their usefulness and usability in real-world contexts. However, given the complexity of learning analytics (LA) systems, it can be challenging to meaningfully involve non-technical stakeholders throughout their design and development. This article reports on the iterative co-design, development, and classroom evaluation of Konscia, a wearable, real-time awareness tool for teachers working in AI-enhanced K-12 classrooms. In the process, we argue that the co-design of LA systems requires new kinds of prototyping methods. We introduce one of our own prototyping methods, REs, to address unique challenges of co-prototyping LA tools. This work presents the first end-to-end demonstration of how non-technical stakeholders can participate throughout the whole design process for a complex LA system—from early generative phases to the selection and tuning of analytics to evaluation in real-world contexts. We conclude by providing methodological recommendations for future LA co-design efforts.