A Randomized Experiment to Compare Conventional, Computerized, and Computerized Adaptive Administration of Ordinal Polytomous Attitude Items

• Published in: Applied psychological measurement Vol. 29; no. 3; pp. 159 - 183
• Main Authors: Hol, A. Michiel, Vorst, Harrie C. M., Mellenbergh, Gideon J.
• Format: Journal Article
• Language: English
• Published: Thousand Oaks, CA Sage Publications 01.05.2005
• Subjects: Adaptive Testing; Attitude Measures; Computer Assisted Testing; High School Students; Item Response Theory; Psychological Studies; School Attitudes; Standardized Tests; statistical validity; test administration; computer assisted testing; Likert scales; item response theory; attitude measurement; adaptive testing
• ISSN: 0146-6216; 1552-3497
• DOI: 10.1177/0146621604271268

A total of 520 high school students were randomly assigned to a paper-and-pencil test (PPT), a computerized standard test (CST), or a computerized adaptive test (CAT) version of the Dutch School Attitude Questionnaire (SAQ), consisting of ordinal polytomous items. The CST administered items in the same order as the PPT. The CAT administered all items of three SAQ subscales in adaptive order using Samejima’s graded response model, so that six different stopping rule settings could be applied afterwards. School marks were used as external criteria. Results showed significant but small multivariate administration mode effects on conventional raw scores and small to medium effects on maximum likelihood latent trait estimates. When the precision of CAT latent trait estimates decreased, correlations with grade point average in general decreased. However, the magnitude of the decrease was not very large as compared to the PPT, the CST, and the CAT without the stopping rule.