Designing Automated Guidance to Promote Productive Revision of Science Explanations

• Published in: International journal of artificial intelligence in education Vol. 27; no. 4; pp. 729 - 757
• Main Authors: Tansomboon, Charissa, Gerard, Libby F., Vitale, Jonathan M., Linn, Marcia C.
• Format: Journal Article
• Language: English
• Published: New York Springer New York 01.12.2017; Springer; Springer Nature B.V
• Subjects: Achievement Gains; Artificial Intelligence; Automation; Beliefs; Classrooms; Comparative Analysis; Computer Science; Computer Uses in Education; Computers; Computers and Education; Design; Educational Technology; Evidence; Guidance; Instructional Effectiveness; Iterative methods; Knowledge; Language Processing; Learning; Meta Analysis; Middle School Students; Middle Schools; Natural Language Processing; Planning; Prior Learning; Revision (Written Composition); Revisions; Science; Science Instruction; Science Teachers; Student Motivation; Student writing; Students; Teaching Methods; Test Construction; User Interfaces and Human Computer Interaction; Writing (Composition); Writing revisions; Automated guidance; Science learning; Motivation; Knowledge integration
• ISSN: 1560-4292; 1560-4306
• DOI: 10.1007/s40593-017-0145-0

Supporting students to revise their written explanations in science can help students to integrate disparate ideas and develop a coherent, generative account of complex scientific topics. Using natural language processing to analyze student written work, we compare forms of automated guidance designed to motivate productive revision and help students integrate their understanding of science. Research shows the benefit of providing timely, transparent guidance to students and identifies some challenges. Specifically, (a) students often believe online guidance is generic rather than adapted to their response; and (b) students do not always engage effortfully with online guidance to improve their written responses. We conducted two studies to address these challenges. In Study 1, we created transparent guidance that clarified how the computer personalizes guidance based on the student response. We hypothesized that transparent guidance would be especially valuable for low prior knowledge students who might expect the computer guidance to be too difficult. We found that transparent guidance had a greater impact than typical guidance on low prior knowledge student revisions, suggesting that student beliefs about how guidance is designed influence their performance. In Study 2, implemented in six schools, we compared two specific guidance strategies: revisiting evidence and planning writing changes. We found that both revisiting and planning guidance resulted in significant improvement in student knowledge integration, although neither guidance strategy showed a significant advantage over the other. In addition, we found that the form of guidance interacted with school, suggesting that teacher practices could reinforce a specific guidance strategy. These results illustrate ways to design guidance to strengthen student understanding of science. They raise important questions about when to encourage revisiting, how to design instruction focused on planning, and how to instill a lifelong practice of engaging in iterative refinement of scientific explanations.