Intent detection using semantically enriched word embeddings

• Published in: 2016 IEEE Spoken Language Technology Workshop (SLT) pp. 414 - 419
• Main Authors: Joo-Kyung Kim, Tur, Gokhan, Celikyilmaz, Asli, Bin Cao, Ye-Yi Wang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.12.2016
• Subjects: Context; Dictionaries; Hafnium; intent detection; Linear programming; LSTM; semantic lexicons; Semantics; spoken language understanding; Training; Vocabulary; word embeddings
• DOI: 10.1109/SLT.2016.7846297

State-of-the-art targeted language understanding systems rely on deep learning methods using 1-hot word vectors or off-the-shelf word embeddings. While word embeddings can be enriched with information from semantic lexicons (such as WordNet and PPDB) to improve their semantic representation, most previous research on word-embedding enriching has focused on improving intrinsic word-level tasks such as word analogy and antonym detection. In this work, we enrich word embeddings to force semantically similar or dissimilar words to be closer or farther away in the embedding space to improve the performance of an extrinsic task, namely, intent detection for spoken language understanding. We utilize several semantic lexicons, such as WordNet, PPDB, and Macmillan Dictionary to enrich the word embeddings and later use them as initial representation of words for intent detection. Thus, we enrich embeddings outside the neural network as opposed to learning the embeddings within the network, and, on top of the embeddings, build bidirectional LSTM for intent detection. Our experiments on ATIS and a real log dataset from Microsoft Cortana show that word embeddings enriched with semantic lexicons can improve intent detection.