Using ChatGPT in Undergraduate Computer Science and Software Engineering Courses: A Students' Perspective

• Published in: Proceedings - Frontiers in Education Conference pp. 1 - 9
• Main Authors: Andersen-Kiel, Noah, Linos, Panagiotis Panos
• Format: Conference Proceeding
• Language: English
• Published: IEEE 13.10.2024
• Subjects: Chatbot; Chatbots; ChatGPT; Codes; Computer bugs; Computer Science Education; EPICS; GenAl; Object oriented programming; Programming profession; Project management; Python; Scalability; Software engineering; Surveys
• ISSN: 2377-634X
• DOI: 10.1109/FIE61694.2024.10892934

This innovative practice full paper presents an empirical study aimed at evaluating the potential of ChatGPT, an advanced AI-driven chatbot, as a supplementary educational tool in undergraduate Computer Science and Software Engineering (CSSE) courses. The study, initiated in the summer of 2023, focused on assessing ChatGPT's capabilities in generating accurate and complete computer code, identifying and rectifying code defects (bugs), and its scalability in handling larger programs. To achieve this, we conducted a series of experiments with ChatGPT. In one experiment, we introduced bugs into small programs from introductory CSSE courses. ChatGPT was tasked with detecting these defects and providing recommendations for fixing them. We evaluated ChatGPT's effectiveness in bug detection, the quality of its recommendations, and the completeness of the proposed solutions. We sought answers to questions such as whether ChatGPT found all injected defects, provided appropriate recommendations, and delivered high-quality solutions based on criteria like code completeness, size, complexity, and readability. In another experiment, ChatGPT was asked to generate code for assignments from previous CSSE courses, including Intro to Computer Science and Programming in C++, Intro to Python Programming, and Object-Oriented Programming and Data Structures using Java. We assessed the generated code's correctness and quality in comparison to student-written code. Similarly, in a third experiment, we evaluated ChatGPT's ability to generate larger programs using requirement specifications from an upper-division CSSE course on Agile Software Engineering. Analyzing both qualitative and quantitative data from these experiments during the summer, we determined that ChatGPT showed promise as an educational tool. Consequently, we developed a plan to integrate ChatGPT into select CSSE courses for the fall semester of 2023. Specifically, ChatGPT was integrated into two of our introductory CSSE courses enabling students to utilize it for debugging assignments and generating practice questions for exam preparation. In addition, we encouraged student teams in our EPICS (Engineering Projects In Community Service) course to utilize ChatGPT as a supplementary aid to help them find and learn any new programming languages or technologies needed for their projects. Anonymous surveys were conducted at the beginning and at the end of these courses to collect feedback from students regarding their experiences with ChatGPT. Initial responses indicated that students were generally familiar with ChatGPT and expressed curiosity about its potential utility, although some skepticism was present. However, by the semester's end, students demonstrated a positive shift in perceptions. They appreciated ChatGPT's assistance in rectifying code bugs, especially after-hours. Additionally, students valued ChatGPT for generating practice questions for exams, despite some inconsistencies in its responses. In our EPICS course, students felt ChatGPT was useful for learning new technologies, though opinions varied on its project management benefits. Lastly, the majority of students felt that ChatGPT helped them adjust to their work progress, showing its potential utility in keeping pace with ongoing projects. Based on student feedback, we propose integrating ChatGPT into future CSSE courses. Finally, as AI -based tools become more integral to academic settings, we believe that disseminating our experiences could potentially enhance engineering and computing education.