Developing individualized feedback for listening assessment: Combining standard setting and cognitive diagnostic assessment approaches

• Published in: Language testing Vol. 39; no. 1; pp. 90 - 116
• Main Authors: Min, Shangchao, He, Lianzhen
• Format: Journal Article
• Language: English
• Published: London, England SAGE Publications 01.01.2022; Sage Publications Ltd
• Subjects: Classification; Cognition; Cognitive Ability; College Students; Critical discourse analysis; Diagnostic Tests; English (Second Language); English as a second language instruction; English as a second language tests; English Language Learners; English proficiency; Feedback; Feedback (Response); Field Tests; Language assessment; Language Proficiency; Language Tests; Listening Comprehension Tests; Mastery Learning; Oral Language; Second Language Instruction; Second Language Learning; Standard Setting; Standard Setting (Scoring); individualized feedback; score report; listening assessment; standard setting; Cognitive diagnosis
• ISSN: 0265-5322; 1477-0946
• DOI: 10.1177/0265532221995475

In this study, we present the development of individualized feedback for a large-scale listening assessment by combining standard setting and cognitive diagnostic assessment (CDA) approaches. We used the performance data from 3,358 students’ item-level responses to a field test of a national EFL test primarily intended for tertiary-level EFL learners. The results showed that proficiency classifications and subskill mastery classifications were generally of acceptable reliability, and the two kinds of classifications were in alignment with each other at individual and group levels. The outcome of the study is a set of descriptors that describe each test taker’s ability to understand certain level of oral texts and his or her cognitive performance. The current study, by illustrating the feasibility of combining standard setting and CDA approaches to produce individualized feedback, contributes to the enhancement of score reporting and addresses the long-standing criticism that large-scale language assessments fail to provide individualized feedback to link assessment with instruction.