Cross-Speaker Style Transfer for Text-to-Speech Using Data Augmentation

• Published in: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 6797 - 6801
• Main Authors: Sam Ribeiro, Manuel, Roth, Julian, Comini, Giulia, Huybrechts, Goeric, Gabrys, Adam, Lorenzo-Trueba, Jaime
• Format: Conference Proceeding
• Language: English
• Published: IEEE 23.05.2022
• Subjects: Acoustics; Conferences; Controllability; cross-speaker; data augmentation; Signal processing; speaking style transfer; Speech processing; text-to-speech
• DOI: 10.1109/ICASSP43922.2022.9746179

We address the problem of cross-speaker style transfer for text-to-speech (TTS) using data augmentation via voice conversion. We assume to have a corpus of neutral non-expressive data from a target speaker and supporting conversational expressive data from different speakers. Our goal is to build a TTS system that is expressive, while retaining the target speaker's identity. The proposed approach relies on voice conversion to first generate high-quality data from the set of supporting expressive speakers. The voice converted data is then pooled with natural data from the target speaker and used to train a single-speaker multi-style TTS system. We provide evidence that this approach is efficient, flexible, and scalable. The method is evaluated using one or more supporting speakers, as well as a variable amount of supporting data. We further provide evidence that this approach allows some controllability of speaking style, when using multiple supporting speakers. We conclude by scaling our proposed technology to a set of 14 speakers across 7 languages. Results indicate that our technology consistently improves synthetic samples in terms of style similarity, while retaining the target speaker's identity.