A little bias goes a long way: the effects of feedback on the strategic regulation of accuracy on formula-scored tests

• Published in: Journal of experimental psychology. Applied Vol. 19; no. 4; p. 383
• Main Authors: Arnold, Michelle M, Higham, Philip A, Martín-Luengo, Beatriz
• Format: Journal Article
• Language: English
• Published: United States 01.12.2013
• Subjects: Bias; Educational Measurement - methods; Educational Measurement - standards; Feedback, Psychological; Humans; Reproducibility of Results; Students - psychology
• ISSN: 1939-2192
• DOI: 10.1037/a0034833

Under formula-scoring rules for multiple-choice exams, a penalty is applied to incorrect responses to reduce noise in the observed score. To avoid the penalty individuals are allowed to "pass," and therefore they must be able to strategically regulate the accuracy of their reporting by deciding which and how many questions to answer. To investigate the effect of bias within this framework, Higham (2007) introduced bias profiles, which show the score obtained under formula scoring (corrected score) as a function of the omission rate. Bias profiles estimate the optimal number of questions that should be answered to maximize the corrected score (i.e., optimal bias). Our initial research showed that individuals tend to be too conservative when setting reporting criteria, "omitting" too many answers. The present three experiments introduced a feedback manipulation whereby participants were informed of the optimal omission rate after completing a test and asked to alter their reporting decisions accordingly. This feedback and concomitant alteration of reporting decisions led to improved corrected scores on true/false (Experiment 1), 2-alternative tests (Experiments 2), and 4-alternative tests (Experiment 3). Importantly, corrected scores at optimal bias also were higher than at forced-report for both true/false and 2-alternative tests. Furthermore, in Experiment 3, feedback based on one test improved scores on a second test, and participants were more likely to perform optimally on a third test without feedback. These effects suggest that optimal-bias feedback may have long-term effects and generalize to new tests.