Attention-based Model for Multi-modal sentiment recognition using Text-Image Pairs

• Published in: 2023 4th International Conference on Innovative Trends in Information Technology (ICITIIT) pp. 1 - 5
• Main Authors: Pandey, Ananya, Vishwakarma, Dinesh Kumar
• Format: Conference Proceeding
• Language: English
• Published: IEEE 11.02.2023
• Subjects: Benchmark testing; BERT; Bidirectional LSTM; Bit error rate; Convolution; Convolution block attention module (CBAM); Information technology; multi-modal sentiment recognition (MSR); Predictive models; Sentiment recognition (SR); Task analysis
• DOI: 10.1109/ICITIIT57246.2023.10068626

Multi-modal sentiment recognition (MSR) is an emerging classification task that aims to categorize sentiment polarities for a given multi-modal dataset. The majority of work done in the past relied heavily on text-based information. However, in many scenarios, text alone is frequently insufficient to predict sentiment accurately; as a result, academics are more motivated to engage in the subject of MSR. In light of this, we proposed an attention-based model for MSR using image-text pairs of tweets. To effectively capture the vital information from both modalities, our approach combines BERT and ConvNet with CBAM (convolution block attention module) attention. The outcomes of our experimentations on the Twitter-17 dataset demonstrate that our method is capable of sentiment classification accuracy that is superior to that of competing approaches.