Predicting Sentiments in Image Advertisements using Semantic Relations among Sentiment Labels

Understanding the sentiments evoked by advertisements is crucial in serving them appropriately to consumers. Advertisements often use images to evoke sentiments. An image can convey multiple sentiments of different nature. Automatically predicting these multiple sentiments can help serve better adve...

Full description

Saved in:
Bibliographic Details
Published in2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) pp. 1640 - 1648
Main Authors Pilli, Stephen, Patwardhan, Manasi, Pedanekar, Niranjan, Karande, Shirish
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2020
Subjects
Convolution
Feature extraction
Mathematical model
Measurement
Neural networks
Semantics
Visualization
Online AccessGet full text

Cover

More Information
Summary:Understanding the sentiments evoked by advertisements is crucial in serving them appropriately to consumers. Advertisements often use images to evoke sentiments. An image can convey multiple sentiments of different nature. Automatically predicting these multiple sentiments can help serve better advertisements to consumers, especially in an online scenario at scale. In this paper, we present a neural network model based on graph convolution to predict such sentiments, which exploits the semantic relationship among the sentiment labels. We use it to predict multiple sentiment labels using an annotated dataset of 30,340 image-based advertisements. We also find a distance metric that best represents the distribution of sentiments in the dataset and utilizes it in a loss function that separates applicable sentiments from the non-applicable ones. We report an improvement in mean average precision and overall F1 score over a multi-modal multi-task state-of-the-art model.
ISSN:2160-7516
DOI:10.1109/CVPRW50498.2020.00212