Reliable estimation of quality scores by a small calibrated listening panel

In this work we propose the calibrated-mean-opinion-score (CMOS), which accounts for varying levels of precision and bias across subjects in a listening test. We adopt the Bayesian statistical framework, where the hyper-parameters of priors are learned via empirical Bayes, and the posterior is appro...

Full description

Saved in:
Bibliographic Details
Published inProceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) pp. 8198 - 8202
Main Authors Mossavat, Iman, Kleijn, W. Bastiaan
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.05.2013
Subjects
Bayes methods
Bayesian inference
bias
Calibration
CMOS integrated circuits
Equations
Listening test
Mathematical model
Speech
Speech quality
Online AccessGet full text

Cover

More Information
Summary:In this work we propose the calibrated-mean-opinion-score (CMOS), which accounts for varying levels of precision and bias across subjects in a listening test. We adopt the Bayesian statistical framework, where the hyper-parameters of priors are learned via empirical Bayes, and the posterior is approximated by a computationally inexpensive variational technique. As our experimental results show, CMOS is more robust to noisy and biased subjects than MOS. As a result, CMOS can be used to improve the reliability of listening test results when a small test panel is used. To correct for the subjects in the test panel, calibration signals are required. Calibration signals are rated by a panel larger than the test panel. The key to saving human labor and cost is that only a few calibration signals are required, and that it is possible to share calibration signals across listening tests.
ISSN:1520-6149
DOI:10.1109/ICASSP.2013.6639263