Target Speaker Extraction with Curriculum Learning

This paper presents a novel approach to target speaker extraction (TSE) using Curriculum Learning (CL) techniques, addressing the challenge of distinguishing a target speaker's voice from a mixture containing interfering speakers. For efficient training, we propose designing a curriculum that s...

Full description

Saved in:
Bibliographic Details
Main Authors Liu, Yun, Liu, Xuechen, Miao, Xiaoxiao, Yamagishi, Junichi
Format Journal Article
LanguageEnglish
Published 11.06.2024
Subjects
Computer Science - Sound
Online AccessGet full text

Cover

More Information
Summary:This paper presents a novel approach to target speaker extraction (TSE) using Curriculum Learning (CL) techniques, addressing the challenge of distinguishing a target speaker's voice from a mixture containing interfering speakers. For efficient training, we propose designing a curriculum that selects subsets of increasing complexity, such as increasing similarity between target and interfering speakers, and that selects training data strategically. Our CL strategies include both variants using predefined difficulty measures (e.g. gender, speaker similarity, and signal-to-distortion ratio) and ones using the TSE's standard objective function, each designed to expose the model gradually to more challenging scenarios. Comprehensive testing on the Libri2talker dataset demonstrated that our CL strategies for TSE improved the performance, and the results markedly exceeded baseline models without CL about 1 dB.
DOI:10.48550/arxiv.2406.07845