Model-Based Approach for Measuring the Fairness in ASR

The issue of fairness arises when the automatic speech recognition (ASR) systems do not perform equally well for all subgroups of the population. In any fairness measurement studies for ASR, the open questions of how to control the nuisance factors, how to handle unobserved heterogeneity across spea...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Liu, Zhe, Irina-Elena Veliche, Peng, Fuchun
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 19.09.2021
Subjects
Automatic speech recognition
Heterogeneity
Subgroups
Online AccessGet full text

Cover

More Information
Summary:The issue of fairness arises when the automatic speech recognition (ASR) systems do not perform equally well for all subgroups of the population. In any fairness measurement studies for ASR, the open questions of how to control the nuisance factors, how to handle unobserved heterogeneity across speakers, and how to trace the source of any word error rate (WER) gap among different subgroups are especially important - if not appropriately accounted for, incorrect conclusions will be drawn. In this paper, we introduce mixed-effects Poisson regression to better measure and interpret any WER difference among subgroups of interest. Particularly, the presented method can effectively address the three problems raised above and is very flexible to use in practical disparity analyses. We demonstrate the validity of proposed model-based approach on both synthetic and real-world speech data.
ISSN:2331-8422