EFL reading test on mobile versus on paper: a study from metacognitive strategy use to test-media impacts

• Published in: Educational assessment, evaluation and accountability Vol. 32; no. 3; pp. 373 - 394
• Main Authors: Zou, Xiao-Ling, Ou, Ling
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 01.08.2020; Springer; Springer Nature B.V
• Subjects: Assessment; Attitudes; Computer Assisted Testing; Dynamic assessment; Education; Educational evaluation; English (Second Language); English as a second language; English as a second language tests; Feedback; Handheld Devices; Individual Differences; Language assessment; Learning Strategies; Metacognition; Psychology; Qualitative research; Quantitative analysis; Reading; Reading Ability; Reading comprehension; Reading Skills; Reading Strategies; Reading Tests; Resistance (Psychology); Second Language Learning; Second language reading; Test Format; Test Results; Testing and Evaluation; Tests; Test scores; Mobile-based reading test; Paper-based reading test; Test-media impact; Metacognitive strategy use
• ISSN: 1874-8597; 1874-8600
• DOI: 10.1007/s11092-020-09320-0

This paper started with the examination of the effect of mobile-based reading test on EFL test-takers of different attitudes from their perception of metacognitive strategy use and the knock-on impact on test results. Then other possible factors of such impact are explored and corresponding solutions are presented. Quantitative analysis of questionnaire and reading test data indicated that test-takers with low acceptance of mobile-based reading test might be adversely affected in their metacognitive strategy use of planning, monitoring, and evaluating, as well as their test results. Qualitative analysis of the relevant literature and the interview data, combined with the quantitative results, revealed that, besides the inferiority of metacognitive regulation on screen, the negative effects might also involve other complicated and intertwined factors, such as reading proficiency, psychology, attitudes, adaptability, habits, and personality, as well as text and question presentation form, which could, in turn, lead to poor performance in this new test form. Therefore, metacognitive training, psychological preparation, guided practice, elaborated feedback, constructive advice, personalized dynamic assessment, optimization of the text and question presentation form, and overcoming of screen inferiority are necessary for better assessment ecology so that test-takers could gain more benefit and convenience brought by this new assessment medium.