Supporting skill integration in an intelligent tutoring system for code tracing

• Published in: Journal of computer assisted learning Vol. 39; no. 2; pp. 477 - 500
• Main Authors: Huang, Yun, Brusilovsky, Peter, Guerra, Julio, Koedinger, Kenneth, Schunn, Christian
• Format: Journal Article
• Language: English
• Published: Chichester, UK John Wiley & Sons, Inc 01.04.2023; Wiley; Wiley Subscription Services, Inc
• Subjects: Achievement Gains; Active learning; adaptive educational systems; Coding; Colleges & universities; Computation; computer science education; Design analysis; domain modelling; Educational Technology; Effect Size; Instructional Design; Intelligent Tutoring Systems; Learning; Mastery Learning; Mathematics education; Programming; programming education; Programming languages; Robustness; Science education; Skill Development; Skills; STEM Education; Students; System effectiveness; Technical education; Technology education; Tracing; Tutoring; Tutors
• ISSN: 0266-4909; 1365-2729
• DOI: 10.1111/jcal.12757

Background Skill integration is vital in students' mastery development and is especially prominent in developing code tracing skills which are foundational to programming, an increasingly important area in the current STEM education. However, instructional design to support skill integration in learning technologies has been limited. Objectives The current work presents the development and empirical evaluation of instructional design targeting students' difficulties in code tracing particularly in integrating component skills in the Trace Table Tutor (T3), an intelligent tutoring system. Methods Beyond the instructional features of active learning, step‐level support, and individualized problem selection of intelligent tutoring systems (ITS), the instructional design of T3 (e.g., hints, problem types, problem selection) was optimized to target skill integration based on a domain model where integrative skills were represented as combinations of component skills. We conducted an experimental study in a university‐level introductory Python programming course and obtained three findings. Results and Conclusions First, the instructional features of the ITS technology support effective learning of code tracing, as evidenced by significant learning gains (medium‐to‐large effect sizes). Second, performance data supports the existence of integrative skills beyond component skills. Third, an instructional design focused on integrative skills yields learning benefits beyond a design without such focus, such as improving performance efficiency (medium‐to‐large effect sizes). Major Takeaways Our work demonstrates the value of designing for skill integration in learning technologies and the effectiveness of the ITS technology for computing education, as well as provides general implications for designing learning technologies to foster robust learning. Lay Description What is currently known about the subject matter? Skill integration is vital in students' mastery development, and is especially prominent in developing coding tracing skills. Coding tracing skills are foundational to programming, an increasingly important area in the current STEM education, yet many novice students struggle with code tracing. Instructional design to support skill integration in learning technologies has been limited. One of the most effective technologies for skill mastery, intelligent tutoring systems (ITS), is underused or only used in a partial form in programming education. What our paper adds to this? We present the development and empirical evaluation of instructional design targeting students' difficulties in code tracing particularly in integrating component skills in an ITS, the Trace Table Tutor. The instructional features of the ITS technology (e.g., active learning, step‐level support, individualized problem selection) support effective learning of code tracing. Students' performance data supports the existence of integrative skills beyond component skills. An instructional design that provides deliberate practice and focused practice on integrative skills yields learning benefits beyond a design without such features. The implications of study findings for practitioners Designing for skill integration is valuable and should receive more attention in designing learning technologies to foster robust learning. The instructional features of full‐scale ITSs are effective in supporting learning and merit further application in learning technologies in computing education.