Position of Correct Option and Distractors Impacts Responses to Multiple-Choice Items: Evidence From a National Test

• Published in: Educational and psychological measurement Vol. 83; no. 5; pp. 861 - 884
• Main Authors: Lions, Séverin, Dartnell, Pablo, Toledo, Gabriela, Godoy, María Inés, Córdova, Nora, Jiménez, Daniela, Lemarié, Julie
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.10.2023; SAGE PUBLICATIONS, INC
• Subjects: Academic Achievement; Accuracy; Achievement tests; College Entrance Examinations; Comparative Analysis; Difficulty Level; Educational tests & measurements; Error of Measurement; Feedback (Response); Foreign Countries; High Stakes Tests; Humanities and Social Sciences; Interference (Learning); Item Analysis; Language Processing; Multiple Choice Tests; Psychology; Reading Processes; Responses; Secondary School Students; Standardized Tests; Test Items; Chile; multiple choice; tests; distractors; academic achievement; response order; educational measurement
• ISSN: 0013-1644; 1552-3888; 1552-3888
• DOI: 10.1177/00131644221132335

Even though the impact of the position of response options on answers to multiple-choice items has been investigated for decades, it remains debated. Research on this topic is inconclusive, perhaps because too few studies have obtained experimental data from large-sized samples in a real-world context and have manipulated the position of both correct response and distractors. Since multiple-choice tests’ outcomes can be strikingly consequential and option position effects constitute a potential source of measurement error, these effects should be clarified. In this study, two experiments in which the position of correct response and distractors was carefully manipulated were performed within a Chilean national high-stakes standardized test, responded by 195,715 examinees. Results show small but clear and systematic effects of options position on examinees’ responses in both experiments. They consistently indicate that a five-option item is slightly easier when the correct response is in A rather than E and when the most attractive distractor is after and far away from the correct response. They clarify and extend previous findings, showing that the appeal of all options is influenced by position. The existence and nature of a potential interference phenomenon between the options’ processing are discussed, and implications for test development are considered.