Federated Marginal Personalization for ASR Rescoring

We introduce federated marginal personalization (FMP), a novel method for continuously updating personalized neural network language models (NNLMs) on private devices using federated learning (FL). Instead of fine-tuning the parameters of NNLMs on personal data, FMP regularly estimates global and pe...

Full description

Saved in:
Bibliographic Details
Published inICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 7388 - 7392
Main Authors Liu, Zhe, Peng, Fuchun
Format Conference Proceeding
LanguageEnglish
Published IEEE 06.06.2021
Subjects
Adaptation models
Artificial neural networks
automatic speech recognition
Conferences
Error analysis
Federated learning
language modeling
personalization
Privacy
second-pass rescoring
Signal processing
Speech recognition
Online AccessGet full text

Cover

More Information
Summary:We introduce federated marginal personalization (FMP), a novel method for continuously updating personalized neural network language models (NNLMs) on private devices using federated learning (FL). Instead of fine-tuning the parameters of NNLMs on personal data, FMP regularly estimates global and personalized marginal distributions of words, and adjusts the probabilities from NNLMs by an adaptation factor that is specific to each word. Our presented approach can overcome the limitations of federated fine-tuning and efficiently learn personalized NNLMs on devices. We study the application of FMP on second-pass ASR rescoring tasks. Experiments on two speech evaluation datasets show modest word error rate (WER) reductions. We also demonstrate that FMP could offer reasonable privacy with only a small decrease in speech recognition accuracy.
ISSN:2379-190X
DOI:10.1109/ICASSP39728.2021.9415056