A Within-Subject Experiment of Item Format Effects on Early Primary Students' Language, Reading, and Numeracy Assessment Results

• Published in: School psychology Vol. 35; no. 1; pp. 80 - 87
• Main Authors: Woodcock, Stuart, Howard, Steven J, Ehrich, John
• Format: Journal Article
• Language: English
• Published: United States American Psychological Association 01.01.2020
• Subjects: Accuracy; Difficulty Level; Elementary School Students; Foreign Countries; Grade 3; Item Response Theory; Literacy; Multiple Choice Tests; National Competency Tests; Numeracy; Reading Comprehension; Standardized Tests; Test Format; Test Items; Australia
• ISSN: 2578-4218; 2578-4226
• DOI: 10.1037/spq0000340

Standardized testing is ubiquitous in educational assessment, but questions have been raised about the extent to which these test scores accurately reflect students' genuine knowledge and skills. To more rigorously investigate this issue, the current study employed a within-subject experimental design to examine item format effects on primary school students' standardized assessment results in literacy, reading comprehension, and numeracy. Eighty-nine Grade 3 students (ages 8-9 years) completed tests that varied only in item format: multiple choice; open-ended; error detection and correction; explain; and, for numeracy questions, low literacy. Analyses contrasted students' performance across these conditions, as well as item response theory-derived item difficulty and ability discrimination estimates. Findings revealed that difficulty increased and accuracy decreased from multiple-choice to open-response to error-correction and explain questions. However, the most difficult item formats tended to yield the greatest discrimination across student ability levels. Despite previous findings to the contrary, low-literacy numeracy questions did not improve student performance or reduce item difficulty. Overall, findings indicated the impact of differing methods of assessment on standardized test performance and highlighted the need for careful consideration of not only the content of assessments but also their approaches to assessment.