A Hybrid Bidirectional Recurrent Convolutional Neural Network Attention-Based Model for Text Classification

• Published in: IEEE access Vol. 7; pp. 106673 - 106685
• Main Authors: Zheng, Jin, Zheng, Limin
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Attention mechanism; bidirectional long short-term memory; Classification; convolutional neural network; Convolutional neural networks; Data mining; Feature extraction; fine-grained sentiment analysis; Machine learning; multi-class text classification; Natural language processing; Neural networks; Recurrent neural networks; Semantics; Sentences; Sentiment analysis; Task analysis; Text categorization
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2932619

The text classification task is an important application in natural language processing. At present, deep learning models, such as convolutional neural network and recurrent neural network, have achieved good results for this task, but the multi-class text classification and the fine-grained sentiment analysis are still challenging. In this paper, we propose a hybrid bidirectional recurrent convolutional neural network attention-based model to address this issue, which named BRCAN. The model combines the bidirectional long short-term memory and the convolutional neural network with the attention mechanism and word2vec to achieve the fine-grained text classification task. In our model, we apply word2vec to generate word vectors automatically and a bidirectional recurrent structure to capture contextual information and long-term dependence of sentences. We also employ a maximum pool layer of convolutional neural network that judges which words play an essential role in text classification, and use the attention mechanism to give them higher weights to capture the key components in texts. We conduct experiments on four datasets, including Yahoo! Answers, Sogou News of the topic classification, Yelp Reviews, and Douban Movies Top250 short reviews of the sentiment analysis. And the experimental results show that the BRCAN outperforms the state-of-the-art models.