A Decade of Discriminative Language Modeling for Automatic Speech Recognition

• Published in: Speech and Computer Vol. 9319; pp. 11 - 22
• Main Authors: Saraclar, Murat, Dikici, Erinc, Arisoy, Ebru
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2015; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Artificial intelligence; Automatic speech recognition; Discriminative training; Information retrieval; Language modeling; Pattern recognition
• ISBN: 3319231316; 9783319231310
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-23132-7_2

This paper summarizes the research on discriminative language modeling focusing on its application to automatic speech recognition (ASR). A discriminative language model (DLM) is typically a linear or log-linear model consisting of a weight vector associated with a feature vector representation of a sentence. This flexible representation can include linguistically and statistically motivated features that incorporate morphological and syntactic information. At test time, DLMs are used to rerank the output of an ASR system, represented as an N-best list or lattice. During training, both negative and positive examples are used with the aim of directly optimizing the error rate. Various machine learning methods, including the structured perceptron, large margin methods and maximum regularized conditional log-likelihood, have been used for estimating the parameters of DLMs. Typically positive examples for DLM training come from the manual transcriptions of acoustic data while the negative examples are obtained by processing the same acoustic data with an ASR system. Recent research generalizes DLM training by either using automatic transcriptions for the positive examples or simulating the negative examples.