The Use of Subsets of Test Questions in an Angoff Standard-Setting Method

• Published in: Educational and psychological measurement Vol. 65; no. 2; pp. 185 - 201
• Main Authors: Ferdous, Abdullah A., Plake, Barbara S.
• Format: Journal Article
• Language: English
• Published: Thousand Oaks, CA SAGE Publications 01.04.2005; Sage Publications; Sage; SAGE PUBLICATIONS, INC
• Subjects: Biological and medical sciences; Fundamental and applied biological sciences. Psychology; Measurement; Probability; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Psychometrics; Psychometrics. Statistics. Methodology; Standard Setting (Scoring); Test Items; Tests; minimum passing score; subsets of test items; stratified item selection; Angoff method; standard setting; Human; Psychometrics
• ISSN: 0013-1644; 1552-3888
• DOI: 10.1177/0013164404264852

In an Angoff standard-setting procedure, judges estimate the probability that a hypothetical randomly selected minimally competent candidate will answer correctly each item constituting the test. In many cases, these item performance estimates are made twice, with information shared with the judges between estimates. Especially for long tests, this estimation process can be time consuming and fatiguing for the participants. This study addressed the possibilities of reducing the number of items individual judges consider in an Angoff standard-setting study. The results of this study suggest that 50% of test items may be sufficient to estimate an equivalent passing score in an Angoff setting study. This could result in substantial savings of time and resources not only for the agencies that have been carrying out this activity but also for the practitioners (judges) who participate in standard-setting studies.