Switch-LSTMs for Multi-Criteria Chinese Word Segmentation

Multi-criteria Chinese word segmentation is a promising but challenging task, which exploits several different segmentation criteria and mines their common underlying knowledge. In this paper, we propose a flexible multi-criteria learning for Chinese word segmentation. Usually, a segmentation criter...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Gong, Jingjing, Chen, Xinchi, Gui, Tao, Qiu, Xipeng
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 19.12.2018
Subjects
Chinese languages
Knowledge management
Multiple criterion
Neural networks
Segmentation
Switching theory
Text editing
Online AccessGet full text

Cover

More Information
Summary:Multi-criteria Chinese word segmentation is a promising but challenging task, which exploits several different segmentation criteria and mines their common underlying knowledge. In this paper, we propose a flexible multi-criteria learning for Chinese word segmentation. Usually, a segmentation criterion could be decomposed into multiple sub-criteria, which are shareable with other segmentation criteria. The process of word segmentation is a routing among these sub-criteria. From this perspective, we present Switch-LSTMs to segment words, which consist of several long short-term memory neural networks (LSTM), and a switcher to automatically switch the routing among these LSTMs. With these auto-switched LSTMs, our model provides a more flexible solution for multi-criteria CWS, which is also easy to transfer the learned knowledge to new criteria. Experiments show that our model obtains significant improvements on eight corpora with heterogeneous segmentation criteria, compared to the previous method and single-criterion learning.
ISSN:2331-8422