Improving cross-lingual low-resource speech recognition by Task-based Meta PolyLoss

• Published in: Computer speech & language Vol. 87; p. 101648
• Main Authors: Chen, Yaqi, Zhang, Hao, Yang, Xukui, Zhang, Wenlin, Qu, Dan
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.08.2024
• Subjects: Low-resource; Meta learning; PolyLoss; Speech recognition; Low-resource; PolyLoss; Speech recognition; Meta learning
• ISSN: 0885-2308; 1095-8363
• DOI: 10.1016/j.csl.2024.101648

Multilingual meta learning has emerged as a promising paradigm for transferring knowledge from source languages to facilitate the learning of low-resource target languages. Loss functions are a type of meta-knowledge that is crucial to the effective training of neural networks. However, the misalignment between the loss functions and the learning paradigms of meta learning degrades the network’s performance. To address this challenge, we propose a new method called Task-based Meta PolyLoss (TMPL) for meta learning. By regarding speech recognition tasks as normal samples and applying PolyLoss to the meta loss function, TMPL can be denoted as a linear combination of polynomial functions based on task query loss. Theoretical analysis shows that TMPL improves meta learning by enabling attention adjustment across different tasks, which can be tailored for different datasets. Experiments on three datasets demonstrated that gradient-based meta learning methods achieve superior performance with TMPL. Furthermore, our experiments validate that the task-based loss function effectively mitigates the misalignment issue. •Multilingual meta-learning ignore the misalignment problem between the loss functions and the learning paradigms of meta learning.•Task-based Meta PolyLoss is a more general meta-loss function, which can be denoted as a linear combination of polynomial functions.•Theoretical analysis shows that TMPL adjust attention across different tasks, which can be tailored for different datasets.•Experiments on three datasets show that verify the effectiveness of TMPL, especially in very low-resource setting.•Experiments analysis shows the task-based loss function effectively mitigates the misalignment issue and is more appropriate for meta-learning.