The automatic assessment of free text answers using a modified BLEU algorithm

• Published in: Computers and education Vol. 56; no. 2; pp. 337 - 345
• Main Authors: Noorbehbahani, F., Kardan, A.A.
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.02.2011; Elsevier
• Subjects: Adult learning; Algorithms; Assessments; Computer Assisted Testing; Education; Educational Assessment; Educational Testing; Electronic Learning; Evaluation methodologies; Evaluation Methods; Learning; Mathematical analysis; Scoring; Semantics; Similarity; Student Evaluation; Students; Texts; Adult learning; Evaluation methodologies
• ISSN: 0360-1315; 1873-782X
• DOI: 10.1016/j.compedu.2010.07.013

e-Learning plays an undoubtedly important role in today’s education and assessment is one of the most essential parts of any instruction-based learning process. Assessment is a common way to evaluate a student’s knowledge regarding the concepts related to learning objectives. In this paper, a new method for assessing the free text answers of students based on the BLEU algorithm is presented. We modify the BLEU algorithm so that it is suitable for assessing free text answers and call the new algorithm the modified BLEU (M-BLEU). To perform an assessment, it is necessary to establish a repository of reference answers written by course instructors or related experts. Several reference answers are included for each question. The M-BLEU algorithm is used to identify the most similar reference answer to a student answer; a similarity score is calculated and applied to score the answers provided by students. Evaluation results show that the proposed method achieves the highest correlation with human expert scores compared to other assessment methods such as latent semantic analysis (LSA) and n-gram co-occurrence.