Test-Time Training for Speech

In this paper, we study the application of Test-Time Training (TTT) as a solution to handling distribution shifts in speech applications. In particular, we introduce distribution-shifts to the test datasets of standard speech-classification tasks -- for example, speaker-identification and emotion-de...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Dumpala, Sri Harsha, Sastry, Chandramouli, Oore, Sageev
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 19.09.2023
Subjects
Algorithms
Background noise
Optimization
Parameters
Speech
Testing time
Training
Online AccessGet full text

Cover

More Information
Summary:In this paper, we study the application of Test-Time Training (TTT) as a solution to handling distribution shifts in speech applications. In particular, we introduce distribution-shifts to the test datasets of standard speech-classification tasks -- for example, speaker-identification and emotion-detection -- and explore how Test-Time Training (TTT) can help adjust to the distribution-shift. In our experiments that include distribution shifts due to background noise and natural variations in speech such as gender and age, we identify some key-challenges with TTT including sensitivity to optimization hyperparameters (e.g., number of optimization steps and subset of parameters chosen for TTT) and scalability (e.g., as each example gets its own set of parameters, TTT is not scalable). Finally, we propose using BitFit -- a parameter-efficient fine-tuning algorithm proposed for text applications that only considers the bias parameters for fine-tuning -- as a solution to the aforementioned challenges and demonstrate that it is consistently more stable than fine-tuning all the parameters of the model.
ISSN:2331-8422