Knowledge-Based Design Analytics for Authoring Courses with Smart Learning Content

• Published in: International journal of artificial intelligence in education Vol. 32; no. 1; pp. 4 - 27
• Main Authors: Albó, Laia, Barria-Pineda, Jordan, Brusilovsky, Peter, Hernández-Leo, Davinia
• Format: Journal Article
• Language: English
• Published: New York Springer New York 01.03.2022; Springer; Springer Nature B.V
• Subjects: Artificial Intelligence; Authoring; Behavioral Objectives; Cognitive Processes; Collaboration; Computer Science; Computer Science Education; Computers and Education; Creating and improving adaptive learning: Smart authoring tools and processes; Data analysis; Data mining; Decision analysis; Decision Making; Design; Design analysis; Difficulty Level; Education; Educational Resources; Educational Technology; Higher Education; Instructional Design; Knowledge; Learning; Learning Activities; Learning Analytics; Learning Processes; Mathematical analysis; Metadata; Pedagogy; Programming; Students; Taxonomy; Teachers; Teaching Methods; User Interfaces and Human Computer Interaction; Visual Aids; Visualization; Knowledge-based analytics; Authoring tool; Blended learning; Design analytics; Concept-level visualization; Smart learning content; Learning design
• DOI: 10.1007/s40593-021-00253-3
• ISSN: 1560-4292

Over the last 10 years, learning analytics have provided educators with both dashboards and tools to understand student behaviors within specific technological environments. However, there is a lack of work to support educators in making data-informed design decisions when designing a blended course and planning appropriate learning activities. In this paper, we introduce knowledge-based design analytics that uncover facets of the learning activities that are being created. A knowledge-based visualization is integrated into edCrumble, a (blended) learning design authoring tool. This new approach is explored in the context of a higher education programming course, where instructors design labs and home practice sessions with online smart learning content on a weekly basis. We performed a within-subjects user study to compare the use of the design tool both with and without visualization. We studied the differences in terms of cognitive load, controllability, confidence and ease of choice, design outcomes, and user actions within the system to compare both conditions with the objective of evaluating the impact of using design analytics during the decision-making phase of course design. Our results indicate that the use of a knowledge-based visualization allows the teachers to reduce the cognitive load (especially in terms of mental demand) and that it facilitates the choice of the most appropriate activities without affecting the overall design time. In conclusion, the use of knowledge-based design analytics improves the overall learning design quality and helps teachers avoid committing design errors.