A closer look at a marginalized test method: Self-assessment as a measure of speaking proficiency

• Published in: Studies in second language acquisition Vol. 45; no. 2; pp. 416 - 441
• Main Authors: Winke, Paula, Zhang, Xiaowan, Pierce, Steven J.
• Format: Journal Article
• Language: English
• Published: New York, USA Cambridge University Press 01.05.2023
• Subjects: Achievement tests; Cost Effectiveness; Energy efficiency; Language acquisition; Language Proficiency; Language Skills; Language tests; Learning Processes; Linguistics; Measurement Techniques; Meta-analysis; Oral Language; Personal Autonomy; Scores; Second Language Instruction; Second Language Learning; Self evaluation; Self Evaluation (Individuals); Spanish as a second language; Spanish language; Standardized Tests; Student Attitudes; Student teacher relationship; Students; Test methods
• ISSN: 0272-2631; 1470-1545
• DOI: 10.1017/S0272263122000079

Second language (L2) teachers may shy away from self-assessments because of warnings that students are not accurate self-assessors. This information stems from meta-analyses in which self-assessment scores on average did not correlate highly with proficiency test results. However, researchers mostly used Pearson correlations, when polyserial could be used. Furthermore, self-assessments today can be computer adaptive. With them, nonlinear statistics are needed to investigate their relationship with other measurements. We wondered, if we explored the relationship between self-assessment and proficiency test scores using more robust measurements (polyserial correlation, continuation-ratio modeling), would we find different results? We had 807 L2-Spanish learners take a computer-adaptive, L2-speaking self-assessment and the ACTFL Oral Proficiency Interview – computer (OPIc). The scores correlated at .61 (polyserial). Using continuation-ratio modeling, we found each unit of increase on the OPIc scale was associated with a 131% increase in the odds of passing the self-assessment thresholds. In other words, a student was more likely to move on to higher self-assessment subsections if they had a higher OPIc rating. We found computer-adaptive self-assessments appropriate for low-stakes L2-proficiency measurements, especially because they are cost-effective, make intuitive sense to learners, and promote learner agency.