Critiquing research published in SSCI and ESCI CALL journals between 2023 and 2024: a research design perspective

• Published in: Journal of China Computer-Assisted Language Learning
• Main Authors: Xiao, Junhong, Peng, Yiwei
• Format: Journal Article
• Language: English
• Published: 07.07.2025
• ISSN: 2748-3479; 2748-3479
• DOI: 10.1515/jccall-2025-0015

CALL publications are not short of systematic literature reviews. The majority of these reviews aim to identify publication trends, research themes, types of technology used, factors influencing technology adoption, task/learning activity design, effectiveness of CALL, theoretical underpinning and so on. No previous publication has focused exclusively on CALL research from the perspective of research design although appropriate design is the first step in ensuring the quality of research outcomes. This study reports on the findings of a research design-focused review of 338 full articles published in five SSCI and ESCI CALL journals between 2023 and 2024. Strengths identified include clear statements of context of study and research purpose, question and/or hypothesis; increasing adoption of mixed-method approach; popularity of experimental research; sound theoretical foundation; multiple data sources/triangulation; specific accounts of data collection and analysis; and acknowledgement of limitations. Aspects which are found to be weak include predominance of micro-level research; scarcity of replication studies; insensitivity to researcher biases; overwhelming dominance of non-probabilistic, convenience sampling; shortage of longitudinal research; undue reliance on self-reported data and unverified and/or unavailable instruments; insufficient attention to ethical issues; over-favoring case study and survey research; and use of ambiguous units of measurement. Implications from the findings are discussed. Broader issues beyond the findings are raised which warrant more attention. By highlighting the strengths and weaknesses of CALL journal publications in terms of research design rather than appraising individual studies, it is hoped that researchers and gatekeepers (editors/reviewers) alike will pay more attention to the design of a study.