Continuous space discriminative language modeling

Discriminative language modeling is a structured classification problem. Log-linear models have been previously used to address this problem. In this paper, the standard dot-product feature representation used in log-linear models is replaced by a non-linear function parameterized by a neural networ...

Full description

Saved in:
Bibliographic Details
Published in2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 2129 - 2132
Main Authors Xu, P., Khudanpur, S., Lehr, M., Prud'hommeaux, E., Glenn, N., Karakos, D., Roark, B., Sagae, K., Saraclar, M., Shafran, I., Bikel, D., Callison-Burch, C., Cao, Y., Hall, K., Hasler, E., Koehn, P., Lopez, A., Post, M., Riley, D.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.03.2012
Subjects
Computer architecture
Discriminative language modeling
Feature extraction
neural network
Neural networks
Speech
Training
Vectors
Online AccessGet full text

Cover

More Information
Summary:Discriminative language modeling is a structured classification problem. Log-linear models have been previously used to address this problem. In this paper, the standard dot-product feature representation used in log-linear models is replaced by a non-linear function parameterized by a neural network. Embeddings are learned for each word and features are extracted automatically through the use of convolutional layers. Experimental results show that as a stand-alone model the continuous space model yields significantly lower word error rate (1% absolute), while having a much more compact parameterization (60%-90% smaller). If the baseline scores are combined, our approach performs equally well.
ISBN:1467300454
9781467300452
ISSN:1520-6149
2379-190X
DOI:10.1109/ICASSP.2012.6288332