SVSNet: An End-to-End Speaker Voice Similarity Assessment Model

Neural evaluation metrics derived for numerous speech generation tasks have recently attracted great attention. In this paper, we propose SVSNet, the first end-to-end neural network model to assess the speaker voice similarity between converted speech and natural speech for voice conversion tasks. U...

Full description

Saved in:
Bibliographic Details
Published inIEEE signal processing letters Vol. 29; pp. 767 - 771
Main Authors Hu, Cheng-Hung, Peng, Yu-Huai, Yamagishi, Junichi, Tsao, Yu, Wang, Hsin-Min
Format Journal Article
LanguageEnglish
Published New York IEEE 2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Coders
Conversion
Correlation
Evaluation
Measurement
Neural evaluation metrics
Neural networks
Predictive models
Representation learning
Similarity
Speech enhancement
Speech recognition
speech similarity assessment
Task analysis
Training
voice conversion
Waveforms
Online AccessGet full text

Cover

More Information
Summary:Neural evaluation metrics derived for numerous speech generation tasks have recently attracted great attention. In this paper, we propose SVSNet, the first end-to-end neural network model to assess the speaker voice similarity between converted speech and natural speech for voice conversion tasks. Unlike most neural evaluation metrics that use hand-crafted features, SVSNet directly takes the raw waveform as input to more completely utilize speech information for prediction. SVSNet consists of encoder, co-attention, distance calculation, and prediction modules and is trained in an end-to-end manner. The experimental results on the Voice Conversion Challenge 2018 and 2020 (VCC2018 and VCC2020) datasets show that SVSNet outperforms well-known baseline systems in the assessment of speaker similarity at the utterance and system levels.
ISSN:1070-9908
1558-2361
DOI:10.1109/LSP.2022.3152672