Measuring Students' Ability to Engage in Scientific Inquiry: A New Instrument to Assess Data Analysis, Explanation, and Argumentation

• Published in: Educational assessment Vol. 25; no. 2; pp. 112 - 135
• Main Authors: Seeratan, Kavita L., McElhaney, Kevin W., Mislevy, Jessica, McGhee Jr, Raymond, Conger, Dylan, Long, Mark C.
• Format: Journal Article
• Language: English
• Published: United States Routledge 01.01.2020; Taylor & Francis Ltd
• Subjects: Biology; Chemistry; Difficulty Level; Educational evaluation; Educational tests & measurements; Goodness of Fit; High School Students; Item Response Theory; Science education; Science Process Skills; Science Tests; Scoring Rubrics; Secondary school students; Student Evaluation; Summative Evaluation; Test Construction; Test Items; Test Reliability; Test Validity
• ISSN: 1062-7197; 1532-6977
• DOI: 10.1080/10627197.2020.1756253

We describe the conceptualization, design, development, validation, and testing of a summative instrument that measures high school students' ability to analyze and evaluate data, construct scientific explanations, and formulate scientific arguments in biology and chemistry disciplinary contexts. Data from 1,405 students were analyzed to evaluate the properties of the instrument. Student measurement separation reliability was 0.71 with items showing satisfactory fit to the Partial Credit Model. The use of the Evidence-Centered Design framework during the design and development process provided a strong foundation for the validity argument. Additional evidence for validation were also gathered. The strengths of the instrument lie in its relatively brief time for administration and a unique approach that integrates science practice and disciplinary knowledge, while simultaneously seeking to decouple their measurement. This research models how to design assessments that align to the National Research Council's framework and informs the design of Next Generation Science Standards-aligned assessments.