Machine learning algorithm for grading open-ended physics questions in Turkish

• Published in: Education and information technologies Vol. 25; no. 5; pp. 3821 - 3844
• Main Authors: Çınar, Ayşe, Ince, Elif, Gezer, Murat, Yılmaz, Özgür
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2020; Springer; Springer Nature B.V
• Subjects: Accuracy; Algorithms; Analysis; Artificial Intelligence; Computer Appl. in Social and Behavioral Sciences; Computer Assisted Testing; Computer Science; Computers and Education; Data mining; Education; Educational evaluation; Educational Technology; Foreign Countries; Grading; Information Systems Applications (incl.Internet); Learning strategies; Machine learning; Multiple Choice Tests; Physics; Science Tests; Scoring; Student Evaluation; Tests; Turkish; Undergraduate Students; User Interfaces and Human Computer Interaction; Turkey; Automatic short answer grading; Machine learning; Short-answer scoring
• ISSN: 1360-2357; 1573-7608
• DOI: 10.1007/s10639-020-10128-0

Worldwide, open-ended questions that require short answers have been used in many exams in fields of science, such as the International Student Assessment Program (PISA), the International Science and Maths Trends Research (TIMSS). However, multiple-choice questions are used for many exams at the national level in Turkey, especially high school and university entrance exams. This study aims to develop an objective and useful automatic scoring model for open-ended questions using machine learning algorithms. Within the scope of this aim, an automated scoring model construction study was conducted on four Physics questions at a University level course with the participation of 246 undergraduate students. The short-answer scoring was handled through an approach that addresses students’ answers in Turkish. Model performing machine learning classification techniques such as SVM (Support Vector Machines), Gini, KNN (k-Nearest Neighbors), and Bagging and Boosting were applied after data preprocessing. The score indicated the accuracy, precision and F1-Score of each predictive model of which the AdaBoost.M1 technique had the best performance. In this paper, we report on a short answer grading system in Turkish, based on a machine learning approach using a constructed dataset about a Physics course in Turkish. This study is also the first study in the field of open-ended exam scoring in Turkish.