Data-Driven Generation of Rubric Parameters from an Educational Programming Environment

• Published in: Artificial Intelligence in Education Vol. 10331; pp. 490 - 493
• Main Authors: Diana, Nicholas, Eagle, Michael, Stamper, John, Grover, Shuchi, Bienkowski, Marie, Basu, Satabdi
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2017; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Alice; Automatic assessment; Educational data mining; Programming
• DOI: 10.1007/978-3-319-61425-0_47
• ISBN: 331961424X; 9783319614243
• ISSN: 0302-9743

We demonstrate that, by using a small set of hand-graded students, we can automatically generate rubric parameters with a high degree of validity, and that a predictive model incorporating these rubric parameters is more accurate than a previously reported model. We present this method as one approach to addressing the often challenging problem of grading assignments in programming environments. A classic solution is creating unit-tests that the student-generated program must pass, but the rigid, structured nature of unit-tests is suboptimal for assessing more open-ended assignments. Furthermore, the creation of unit-tests requires predicting the various ways a student might correctly solve a problem – a challenging and time-intensive process. The current study proposes an alternative, semi-automated method for generating rubric parameters using low-level data from the Alice programming environment.