Evaluating automatic speech recognition-based language learning systems: a case study

• Published in: Computer assisted language learning Vol. 29; no. 4; pp. 833 - 851
• Main Authors: van Doremalen, Joost, Boves, Lou, Colpaert, Jozef, Cucchiarini, Catia, Strik, Helmer
• Format: Journal Article
• Language: English
• Published: Oxford Routledge 18.05.2016; Taylor & Francis Ltd
• Subjects: Assistive Technology; Automatic speech recognition; Belgium; Case Studies; Computer Assisted Instruction; Computer-assisted language learning; computer-assisted pronunciation training; Courseware; Feedback (Response); Foreign Countries; Indo European Languages; Language; Language Acquisition; Learning; Learning Processes; Likert Scales; Machine learning; Morphology; Morphology (Languages); Netherlands; Positive Attitudes; Pronunciation; Pronunciation Instruction; Questionnaires; Research Tools; Second Language Instruction; Second Language Learning; Semi Structured Interviews; Speech; Speech recognition; Student Attitudes; Students; Syntax; Teaching Methods; Usability; Voice recognition; Voice response technology; Belgium; Netherlands
• ISSN: 0958-8221; 1744-3210
• DOI: 10.1080/09588221.2016.1167090

The purpose of this research was to evaluate a prototype of an automatic speech recognition (ASR)-based language learning system that provides feedback on different aspects of speaking performance (pronunciation, morphology and syntax) to students of Dutch as a second language. We carried out usability reviews, expert reviews and user tests to gain insight into the potential of this prototype and the possible ways in which it could be further adapted or improved, with a view to developing specific language learning products. The evaluation revealed that domain experts and users (teachers and students) are generally positive about the system and intend to use it if they get the opportunity. In addition, recommendations have been made which range from specific changes and additions to the system to more general statements about the pedagogical and technological issues involved. These recommendations can be useful to improve this prototype and to develop other ASR-based systems, which can be deployed either as language courseware or as research tools to investigate design hypotheses and language acquisition processes.