Designing and Validating Assessments of Complex Thinking in Science

• Published in: Theory into practice Vol. 54; no. 3; pp. 238 - 254
• Main Authors: Ryoo, Kihyun, Linn, Marcia C.
• Format: Journal Article
• Language: English
• Published: Columbus Routledge 03.07.2015; Taylor & Francis Group; Ohio State University, College of Education
• Subjects: Academic standards; Biological Sciences; Computer Assisted Testing; Educational evaluation; Energy; Inquiry method; Instructional design; Middle School Students; Motivation; Pedagogy; Psychometrics; Quantitative psychology; Reasoning; Science education; Science Instruction; Science Process Skills; Scoring Rubrics; Secondary School Science; Semi Structured Interviews; Teachers; Teaching methods; Thinking Skills; Validity
• ISSN: 0040-5841; 1543-0421
• DOI: 10.1080/00405841.2015.1044374

Typical assessment systems often measure isolated ideas rather than the coherent understanding valued in current science classrooms. Such assessments may motivate students to memorize, rather than to use new ideas to solve complex problems. To meet the requirements of the Next Generation Science Standards, instruction needs to emphasize sustained investigations, and assessments need to create a detailed picture of students' conceptual understanding and reasoning processes. This article describes the design process and potential for automated scoring of 2 forms of inquiry assessment: Energy Stories and MySystem. To design these assessments, we formed a partnership of teachers, discipline experts, researchers, technologists, and psychometricians to align curriculum, assessments, and rubrics. We illustrate how these items document middle school students' reasoning about energy flow in life science. We used evidence from review by science teachers and experts in the discipline; classroom experiments; and psychometric analysis to validate the assessments, rubrics, and automated scoring.