Sentiment Classification Based on Part-of-Speech and Self-Attention Mechanism

• Published in: IEEE access Vol. 8; pp. 16387 - 16396
• Main Authors: Cheng, Kefei, Yue, Yanan, Song, Zhiwen
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classification; Deep learning; Feature extraction; focal loss; Neural networks; Part-of-speech; self-attention mechanism; Semantics; Sentiment analysis; sentiment classification; Speech; Task analysis; Words (language)
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2020.2967103

Currently, various attention-based neural networks have achieved successes in sentiment classification tasks, as attention mechanism is capable of focusing on those words contributing more to the sentiment polarity prediction than others. However, the major drawback of these approaches is that they only pay attention to the words, the sentimental information contained in the part-of-speech(POS) is ignored. To address this problem, in this paper, we propose Part-of-Speech based Transformer Attention Network(pos-TAN). This model not only uses the Self-Attention mechanism to learn the feature expression of the text but also incorporates the POS-Attention, which uses to capture sentimental information contained in part-of-speech. In addition, our innovative introduction of the Focal Loss effectively alleviates the impact of sample imbalance on model performance. We conduct substantial experiments on various datasets, and the encouraging results indicate the efficacy of our proposed approach.