The role of Rasch analysis when conducting science education research utilizing multiple-choice tests

• Published in: Science education (Salem, Mass.) Vol. 90; no. 2; pp. 253 - 269
• Main Authors: Boone, William J, Scantlebury, Kathryn
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc., A Wiley Company 01.03.2006; John Wiley & Sons, Inc; Wiley; Wiley Periodicals Inc
• Subjects: Academic Standards; Achievement Tests; Curriculum subjects: programmes and methods; Data Analysis; Datenanalyse; Education reform; Educational Change; Educational Research; Educational sciences; Ergebnis; Exact and natural sciences; High Achievement; International Studies; Kritik; Leistungsmessung; Measurement Techniques; Multiple Choice Tests; Multiple-Choice-Verfahren; Psychometrics; Rasch Analysis; Science Achievement; Science Education; Science Tests; Teaching methods; Test Bias; Test Reliability; Test Validity; United States; Untersuchung; Verfahren; Vergleichsuntersuchung; United States; USA; Measurement; Docimology; Evaluation; Educational Research; Science Education; Research Method
• ISSN: 0036-8326; 1098-237X
• DOI: 10.1002/sce.20106

Recent international studies note that countries whose students perform well on international science assessments report the need to change science education. Some countries use assessments for diagnostic purposes to assist teachers in addressing their students' needs. However, in the United States, standards‐based reform has focused the national discussion on documenting students' attainment of high educational standards. Students' science achievement is one of those standards, and in many states, “high‐stakes” tests determine the resultant achievement measures. Policymakers and administrators use those tests to rank school performance, to prevent students' graduation, and to evaluate teachers. With science test measures used in different ways, statistical confidence in the measures' validity and reliability is essential. Using a science achievement test from one state's systemic reform project as an example, this paper discusses the strengths of the Rasch model as a psychometric tool and analysis technique, referring to person item maps, anchoring, differential item functioning, and person item fit. Furthermore, the paper proposes that science educators should carefully inspect the tools they use to measure and document changes in educational systems. © 2005 Wiley Periodicals, Inc. Sci Ed 90:253–269, 2006