Channel-Aware Pretraining of Joint Encoder-Decoder Self-Supervised Model for Telephonic-Speech ASR

This paper proposes a novel technique to obtain better downstream ASR performance from a joint encoder-decoder self-supervised model when trained with speech pooled from two different channels (narrow and wide band). The joint encoder-decoder self-supervised model extends the HuBERT model with a Tra...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Sukhadia, Vrunda N, Arunkumar, A, Umesh, S
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 03.06.2023
Subjects
Channels
Clustering
Coders
Encoders-Decoders
Speech
Online AccessGet full text

Cover

More Information
Summary:This paper proposes a novel technique to obtain better downstream ASR performance from a joint encoder-decoder self-supervised model when trained with speech pooled from two different channels (narrow and wide band). The joint encoder-decoder self-supervised model extends the HuBERT model with a Transformer decoder. HuBERT performs clustering of features and predicts the class of every input frame. In simple pooling, which is our baseline, there is no way to identify the channel information. To incorporate channel information, we have proposed non-overlapping cluster IDs for speech from different channels. Our method gives a relative improvement of ~4% over the joint encoder-decoder self-supervised model built with simple pooling of data, which serves as our baseline.
ISSN:2331-8422