Design and Discovery in Educational Assessment: Evidence-Centered Design, Psychometrics, and Educational Data Mining

• Published in: Journal of educational data mining Vol. 4; no. 1; pp. 11 - 48
• Main Authors: Mislevy, Robert J, Behrens, John T, Dicerbo, Kristen E, Levy, Roy
• Format: Journal Article
• Language: English
• Published: International Working Group on Educational Data Mining 2012
• Subjects: Blended Learning; Computer Assisted Testing; Computer Games; Computer Networks; Computer Simulation; Data Analysis; Data Processing; Educational Assessment; Evidence; Information Technology; Item Response Theory; Pattern Recognition; Program Design; Program Implementation; Psychometrics; Technological Advancement; Technology Uses in Education; Video Games
• ISSN: 2157-2100; 2157-2100

"Evidence-centered design" (ECD) is a comprehensive framework for describing the conceptual, computational and inferential elements of educational assessment. It emphasizes the importance of articulating inferences one wants to make and the evidence needed to support those inferences. At first blush, ECD and "educational data mining" (EDM) might seem in conflict: structuring situations to evoke particular kinds of evidence, versus discovering meaningful patterns in available data. However, a dialectic between the two stances increases understanding and improves practice. We first introduce ECD and relate its elements to the broad range of digital inputs relevant to modern assessment. We then discuss the relation between EDM and psychometric activities in educational assessment. We illustrate points with examples from the "Cisco Networking Academy", a global program in which information technology is taught through a blended program of face-to-face classroom instruction, an online curriculum, and online assessments.