Improving Fairness and Robustness in End-to-End Speech Recognition through unsupervised clustering

The challenge of fairness arises when Automatic Speech Recognition (ASR) systems do not perform equally well for all sub-groups of the population. In the past few years there have been many improvements in overall speech recognition quality, but without any particular focus on advancing Equality and...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Irina-Elena Veliche, Fung, Pascale
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 06.06.2023
Subjects
Automatic speech recognition
Clustering
Privacy
Robustness
Training
User groups
Voice recognition
Online AccessGet full text

Cover

More Information
Summary:The challenge of fairness arises when Automatic Speech Recognition (ASR) systems do not perform equally well for all sub-groups of the population. In the past few years there have been many improvements in overall speech recognition quality, but without any particular focus on advancing Equality and Equity for all user groups for whom systems do not perform well. ASR fairness is therefore also a robustness issue. Meanwhile, data privacy also takes priority in production systems. In this paper, we present a privacy preserving approach to improve fairness and robustness of end-to-end ASR without using metadata, zip codes, or even speaker or utterance embeddings directly in training. We extract utterance level embeddings using a speaker ID model trained on a public dataset, which we then use in an unsupervised fashion to create acoustic clusters. We use cluster IDs instead of speaker utterance embeddings as extra features during model training, which shows improvements for all demographic groups and in particular for different accents.
ISSN:2331-8422