AGADIR: Towards Array-Geometry Agnostic Directional Speech Recognition

Wearable devices like smart glasses are approaching the compute capability to seamlessly generate real-time closed captions for live conversations. We build on our recently introduced directional Automatic Speech Recognition (ASR) for smart glasses that have microphone arrays, which fuses multi-chan...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Lin, Ju, Moritz, Niko, Huang, Yiteng, Xie, Ruiming, Sun, Ming, Fuegen, Christian, Seide, Frank
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 18.01.2024
Subjects
Arrays
Automatic speech recognition
Beamforming
Crosstalk
Eyewear
Geometric accuracy
Microphones
Training
Voice recognition
Wearable technology
Online AccessGet full text

Cover

More Information
Summary:Wearable devices like smart glasses are approaching the compute capability to seamlessly generate real-time closed captions for live conversations. We build on our recently introduced directional Automatic Speech Recognition (ASR) for smart glasses that have microphone arrays, which fuses multi-channel ASR with serialized output training, for wearer/conversation-partner disambiguation as well as suppression of cross-talk speech from non-target directions and noise. When ASR work is part of a broader system-development process, one may be faced with changes to microphone geometries as system development progresses. This paper aims to make multi-channel ASR insensitive to limited variations of microphone-array geometry. We show that a model trained on multiple similar geometries is largely agnostic and generalizes well to new geometries, as long as they are not too different. Furthermore, training the model this way improves accuracy for seen geometries by 15 to 28\% relative. Lastly, we refine the beamforming by a novel Non-Linearly Constrained Minimum Variance criterion.
ISSN:2331-8422