AB-LaBSE: Uyghur Sentiment Analysis via the Pre-Training Model with BiLSTM

• Published in: Applied sciences Vol. 12; no. 3; p. 1182
• Main Authors: Pei, Yijie, Chen, Siqi, Ke, Zunwang, Silamu, Wushour, Guo, Qinglang
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.02.2022
• Subjects: Accuracy; BiLSTM; cross-lingual pre-trained language model; data augmentation; Datasets; Deep learning; Dictionaries; Language; low-resource; Machine learning; Methods; Neural networks; Semantics; Sentiment analysis; Sparsity
• DOI: 10.3390/app12031182
• ISSN: 2076-3417

In recent years, more and more attention has been paid to text sentiment analysis, which has gradually become a research hotspot in information extraction, data mining, Natural Language Processing (NLP), and other fields. With the gradual popularization of the Internet, sentiment analysis of Uyghur texts has great research and application value in online public opinion. For low-resource languages, most state-of-the-art systems require tens of thousands of annotated sentences to get high performance. However, there is minimal annotated data available about Uyghur sentiment analysis tasks. There are also specificities in each task—differences in words and word order across languages make it a challenging problem. In this paper, we present an effective solution to providing a meaningful and easy-to-use feature extractor for sentiment analysis tasks: using the pre-trained language model with BiLSTM layer. Firstly, data augmentation is carried out by AEDA (An Easier Data Augmentation), and the augmented dataset is constructed to improve the performance of text classification tasks. Then, a pretraining model LaBSE is used to encode the input data. Then, BiLSTM is used to learn more context information. Finally, the validity of the model is verified via two categories datasets for sentiment analysis and five categories datasets for emotion analysis. We evaluated our approach on two datasets, which showed wonderful performance compared to some strong baselines. We close with an overview of the resources for sentiment analysis tasks and some of the open research questions. Therefore, we propose a combined deep learning and cross-language pretraining model for two low resource expectations.