Preference-based training framework for automatic speech quality assessment using deep neural network

One objective of Speech Quality Assessment (SQA) is to estimate the ranks of synthetic speech systems. However, recent SQA models are typically trained using low-precision direct scores such as mean opinion scores (MOS) as the training objective, which is not straightforward to estimate ranking. Alt...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Cheng-Hung, Hu, Yasuda, Yusuke, Toda, Tomoki
Format Paper Journal Article
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 29.08.2023
Subjects
Artificial neural networks
Correlation coefficients
Quality assessment
Ranking
Speech
Training
Online AccessGet full text

Cover

More Information
Summary:One objective of Speech Quality Assessment (SQA) is to estimate the ranks of synthetic speech systems. However, recent SQA models are typically trained using low-precision direct scores such as mean opinion scores (MOS) as the training objective, which is not straightforward to estimate ranking. Although it is effective for predicting quality scores of individual sentences, this approach does not account for speech and system preferences when ranking multiple systems. We propose a training framework of SQA models that can be trained with only preference scores derived from pairs of MOS to improve ranking prediction. Our experiment reveals conditions where our framework works the best in terms of pair generation, aggregation functions to derive system score from utterance preferences, and threshold functions to determine preference from a pair of MOS. Our results demonstrate that our proposed method significantly outperforms the baseline model in Spearman's Rank Correlation Coefficient.
ISSN:2331-8422
DOI:10.48550/arxiv.2308.15203