A communicative validation study on an English listening test in Korea

• Published in: Language Testing in Asia Vol. 13; no. 1; pp. 26 - 20
• Main Authors: Zhang, Ci, Xu, XiaoShu, Zhang, Yunfeng
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.12.2023; Springer; Springer Nature B.V; SpringerOpen
• Subjects: Academic Ability; Assessment; Classical test theory; College Entrance Examinations; Communicative competence; Communicative Competence (Languages); Communicative validation; Comparative Analysis; Construct Validity; Correlation; Education; Elementary school students; English (Second Language); English as a second language instruction; English as a second language tests; Factor Analysis; Foreign Countries; High School Students; Item Analysis; Item Response Theory; Language Education; Language Testing in China: Space between theory and practice; Language Tests; Listening comprehension; Listening comprehension sub-skills; Listening Comprehension Tests; Listening Skills; Second Language Instruction; Second Language Learning; Secondary school students; Statistical analysis; Summative Evaluation; Test Items; Test Theory; Test validity and reliability; Testing and Evaluation; Validity; South Korea; Factor analysis; Communicative validation; Classical test theory; Listening comprehension sub-skills
• ISSN: 2229-0443; 2229-0443
• DOI: 10.1186/s40468-023-00238-0

This study presents the validation process of a listening test based on a communicative language test proposed by Bachman (Fundamental considerations in language testing, 1990). It was administered to third-grade high school students by the sixteen Korean Provincial Offices of Education for Curriculum and Evaluation in September 2012 to assess their listening ability at the end of high school learning and compare it with the standard of the CSAT (College Scholastic Ability Test). The research questions were the following. First, to what extent does the test measure the listening comprehension construct? Second, what sub-skills does the test measure? Third, to what extent does the test measure communicative ability? To answer these three questions, a study was designed to examine the test’s construct validity using classical test theory (CTT). Then, an exploratory factor analysis (EFA) and a confirmatory factor analysis (CFA) were applied to find a model that would fit the score of 400 examinees to explain the correlation between six divisible sub-skills of English listening comprehension and twenty listening items. R-program is used as a tool to analyze the above data. The results show that this test is not discriminatory, as the purpose of a summative assessment is not to level students into different groups. With an acceptable measurement of the construct, this research concludes that the test does not have a clear division of listening sub-skills but, on the other hand, sufficiently measures communicative ability.