Leveraging computer vision for adaptive learning in STEM education: effect of engagement and self-efficacy

• Published in: International Journal of Educational Technology in Higher Education Vol. 20; no. 1; pp. 53 - 26
• Main Authors: Wu, Ting-Ting, Lee, Hsin-Yu, Wang, Wei-Sheng, Lin, Chia-Ju, Huang, Yueh-Min
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.12.2023; BioMed Central, Ltd; Springer Nature B.V; SpringerOpen
• Subjects: Adaptive learning; Artificial Intelligence; Computer Appl. in Social and Behavioral Sciences; Computer Assisted Instruction; Computer Science; Computer vision; Computers; Computers and Education; Educational Technology; Effectiveness; Engagement assessment; Higher Education; Humanities; ICAP framework; Identification; Information Systems Applications (incl.Internet); Instructional Materials; Interaction; Law; Learner Engagement; Learning; Learning Processes; Mathematics education; Problem Solving; Research Article; Science education; Self Efficacy; Statistics for Social Sciences; STEM Education; Student Evaluation; Technical education; Technology education; Adaptive learning; Computer vision; STEM education; Engagement assessment; Self-efficacy; ICAP framework
• DOI: 10.1186/s41239-023-00422-5
• ISSN: 2365-9440

In the field of Science, Technology, Engineering, and Mathematics (STEM) education, which aims to cultivate problem-solving skills, accurately assessing learners' engagement remains a significant challenge. We present a solution to this issue with the Real-time Automated STEM Engagement Detection System (RASEDS). This innovative system capitalizes on the power of artificial intelligence, computer vision, and the Interactive, Constructive, Active, and Passive (ICAP) framework. RASEDS uses You Only Learn One Representation (YOLOR) to detect and map learners' interactions onto the four levels of engagement delineated in the ICAP framework. This process informs the system's recommendation of adaptive learning materials, designed to boost both engagement and self-efficacy in STEM activities. Our study affirms that RASEDS accurately gauges engagement, and that the subsequent use of these adaptive materials significantly enhances both engagement and self-efficacy. Importantly, our research suggests a connection between elevated self-efficacy and increased engagement. As learners become more engaged in their learning process, their confidence is bolstered, thereby augmenting self-efficacy. We underscore the transformative potential of AI in facilitating adaptive learning in STEM education, highlighting the symbiotic relationship between engagement and self-efficacy.