Checkbox grading of handwritten mathematics exams with multiple assessors: how do students react to the resulting atomic feedback? A mixed-method study

• Published in: ZDM Vol. 56; no. 4; pp. 757 - 773
• Main Authors: Moons, Filip, Iannone, Paola, Vandervieren, Ellen
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.08.2024; Springer Nature B.V
• Subjects: Assessment Literacy; Automation; Check Lists; Cognitive Processes; Computers; Designers; Education; Feedback; Flow Charts; Grading; Handwriting; Interrater Reliability; Learning; Literacy; Literature reviews; Mathematics; Mathematics Education; Mathematics Tests; Original Paper; Questionnaires; Secondary School Mathematics; Student Attitudes; Students; Ticks; Summative feedback; Feedback; Engagement with feedback; Cognitive processing; Computer-assisted assessment; Atomic feedback; Student responses; State examinations; Receptivity
• ISSN: 1863-9690; 1863-9704
• DOI: 10.1007/s11858-024-01550-6

Handwritten tasks are better suited than digital ones to assess higher-order mathematics skills, as students can express themselves more freely. However, maintaining reliability and providing feedback can be challenging when assessing high-stakes, handwritten mathematics exams involving multiple assessors. This paper discusses a new semi-automated grading approach called ‘checkbox grading’. Checkbox grading gives each assessor a list of checkboxes consisting of feedback items for each task. The assessor then ticks those feedback items which apply to the student’s solution. Dependencies between the checkboxes can be set to ensure all assessors take the same route on the grading scheme. The system then automatically calculates the grade and provides atomic feedback to the student, giving a detailed insight into what went wrong and how the grade was obtained. Atomic feedback consists of a set of format requirements for mathematical feedback items, which has been shown to increase feedback’s reusability. Checkbox grading was tested during the final high school mathematics exam (grade 12) organised by the Flemish Exam Commission, with 60 students and 10 assessors. This paper focuses on students’ perceptions of the received checkbox grading feedback and how easily they interpreted it. After the exam was graded, all students were sent an online questionnaire, including their personalised exam feedback. The questionnaire was filled in by 36 students, and 4 of them participated in semi-structured interviews. Findings suggest that students could interpret the feedback from checkbox grading well, with no correlation between students’ exam scores and feedback understanding. Therefore, we suggest that checkbox grading is an effective way to provide feedback, also for students with shaky subject matter knowledge.