C‐BEAM: A confidence‐based evaluation of MCQs for providing feedback to instructors

• Published in: Computer applications in engineering education Vol. 27; no. 1; pp. 112 - 127
• Main Authors: Aleem, Abdul, Gore, Manoj M.
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc 01.01.2019
• Subjects: association rule mining; Beamforming; confidence‐based evaluation; Data mining; educational data mining; Feedback; Marking; marking schemes for MCQs; Multiple choice; multiple‐choice questions; Students; Teachers
• ISSN: 1061-3773; 1099-0542
• DOI: 10.1002/cae.22061

Multiple‐choice questions (MCQs) examination is the most preferred methodology for evaluating knowledge in a specific context for a large number of students. The problem with MCQs is that students apply probabilistic approach to guess answers for scoring marks. Depending on the accuracy of the prediction, it may happen that the examinees who have little or no knowledge score more than or equal to the examinees who have better knowledge. Unexpected failures of the candidates who have passed MCQ based prestigious competitive examinations have also been observed. To overcome these contradictions, this article proposes a novel Confidence‐Based Evaluation Approach for MCQs (C‐BEAM) that curb the use of probabilistic prediction. The proposed marking scheme has been compared with well‐known marking schemes like no‐negative marking, one‐fourth negative marking, and one‐third negative marking. These marking schemes have been applied for the evaluation of eight MCQ quizzes on three engineering subjects that were attempted by more than 500 students at MNNIT Allahabad, India. The analysis of results proves C‐BEAM marking scheme as more effective than other compared methods in minimizing the effect of guessed answers. In addition to the effectiveness, C‐BEAM forms the basis for the confidence‐based feedback of students. This article also applies association rule mining methods to the evaluation data generated from C‐BEAM method for discovering association rules. These rules are confidence‐based feedback of students that would help instructors to improve the teaching process.