Computer-Assisted Assessment in Open-Ended Activities through the Analysis of Traces: A Proof of Concept in Statistics with R Commander

• Published in: Eurasia Journal of Mathematics, Science and Technology Education Vol. 15; no. 9; p. em1743
• Main Authors: Calvo, Miquel, Carnicer, Artur, Cuadros, Jordi, Martori, Francesc, Miñarro, Antonio, Serrano, Vanessa
• Format: Journal Article
• Language: English
• Published: East Sussex Modestum 2019
• Subjects: Anàlisi de traces; Automation; Bioinformatics; Bioinformàtica; Comparative Analysis; Computer Assisted Testing; Correlation; Estadística; Foreign Countries; Grading; Introductory Courses; Learning Activities; Learning Analytics; Learning management systems; Online instruction; Problem Solving; Programming Languages; Research Universities; Statistics; Statistics Education; STEM Education; Students; Teaching; Time Management; Trace analysis; Tutoring; Undergraduate Students; User interface; Spain
• ISSN: 1305-8223; 1305-8215; 1305-8223
• DOI: 10.29333/ejmste/108456

Open-ended tasks are common in Science, Technology, Engineering and Mathematics (STEM) education. However, as far as we know, no tools have been developed to assist in the assessment of the solution process of open-ended questions. In this paper, we propose the use of analysis of traces as a tool to address this need. To illustrate this approach, we developed a modified version of R Commander that collects traces of students’ actions and described a way to analyze them by using regular expressions. We used this tool in an undergraduate introductory statistics course. The traces were analyzed by comparing them to predefined problem-solving steps, arranged by the instructor. The analyses provide information about the time students spent on the activity, their work intensity and the choices they made when solving open-ended questions. This automated assessment tool provides grades highly correlated to those obtained by a traditional test and traditional grading scheme.