The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers
Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Previous research has shown that novices can encounter multiple metacognitive difficulties while programming. Novices are typically unaware of how these difficulties are hin...
Saved in:
Main Authors | , , , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
27.05.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Novice programmers often struggle through programming problem solving due to
a lack of metacognitive awareness and strategies. Previous research has shown
that novices can encounter multiple metacognitive difficulties while
programming. Novices are typically unaware of how these difficulties are
hindering their progress. Meanwhile, many novices are now programming with
generative AI (GenAI), which can provide complete solutions to most
introductory programming problems, code suggestions, hints for next steps when
stuck, and explain cryptic error messages. Its impact on novice metacognition
has only started to be explored. Here we replicate a previous study that
examined novice programming problem solving behavior and extend it by
incorporating GenAI tools. Through 21 lab sessions consisting of participant
observation, interview, and eye tracking, we explore how novices are coding
with GenAI tools. Although 20 of 21 students completed the assigned programming
problem, our findings show an unfortunate divide in the use of GenAI tools
between students who accelerated and students who struggled. Students who
accelerated were able to use GenAI to create code they already intended to make
and were able to ignore unhelpful or incorrect inline code suggestions. But for
students who struggled, our findings indicate that previously known
metacognitive difficulties persist, and that GenAI unfortunately can compound
them and even introduce new metacognitive difficulties. Furthermore, struggling
students often expressed cognitive dissonance about their problem solving
ability, thought they performed better than they did, and finished with an
illusion of competence. Based on our observations from both groups, we propose
ways to scaffold the novice GenAI experience and make suggestions for future
work. |
---|---|
DOI: | 10.48550/arxiv.2405.17739 |