Subject-Specialized Chatbot in Higher Education as a Tutor for Autonomous Exam Preparation: Analysis of the Impact on Academic Performance and Students’ Perception of Its Usefulness

• Published in: Education sciences Vol. 15; no. 1; p. 26
• Main Author: Sánchez-Vera, Fulgencio
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.01.2025
• Subjects: Academic achievement; Access to education; Active Learning; Analysis; Artificial intelligence; Automation; Chatbots; Computational linguistics; Customization; Digital technology; Distance learning; Early childhood education; Education; Education, Higher; educational chatbot; Educational Environment; Educational Facilities Improvement; Educational Strategies; educational technology; Electronic Learning; Feedback; Higher education; Independent study; Individual Needs; Influence of Technology; Language processing; Learner Engagement; Learning Experience; Learning Motivation; Learning Processes; Methods; Natural language interfaces; Online Courses; Online instruction; Outcomes of Education; Pedagogy; Perceptions; Personalized learning; Privacy; prompt engineering; Retention (Psychology); School Holding Power; Self Management; Student Motivation; Student Records; Students; Surveys; Teaching methods; Technology Acceptance Model; Technology adoption; Test preparation; Young Children
• ISSN: 2227-7102; 2227-7102
• DOI: 10.3390/educsci15010026

This study evaluates the impact of an AI chatbot as a support tool for second-year students in the Bachelor’s Degree in Early Childhood Education program during final exam preparation. Over 1-month, 42 students used the chatbot, generating 704 interactions across 186 conversations. The study aimed to assess the chatbot’s effectiveness in resolving specific questions, enhancing concept comprehension, and preparing for exams. Methods included surveys, in-depth interviews, and analysis of chatbot interactions. Results showed that the chatbot was highly effective in clarifying doubts (91.4%) and aiding concept understanding (95.7%), although its perceived usefulness was lower in content review (42.9%) and exam simulations (45.4%). Students with moderate chatbot use achieved better academic outcomes, while excessive use did not lead to further improvements. The study also identified challenges in students’ ability to formulate effective questions, limiting the chatbot’s potential in some areas. Overall, the chatbot was valued for fostering study autonomy, though improvements are needed in features supporting motivation and study organization. These findings highlight the potential of chatbots as complementary learning tools but underscore the need for better user training in “prompt engineering” to maximize their effectiveness.