Image retrieval via topic modelling of Instagram hashtags

• Published in: 2020 15th International Workshop on Semantic and Social Media Adaptation and Personalization (SMA pp. 1 - 8
• Main Author: Tsapatsoulis, Nicolas
• Format: Conference Proceeding
• Language: English
• Published: IEEE 29.10.2020
• Subjects: Adaptation models; automatic image annotation; Image annotation; Image retrieval; Instagram hashtags; learning by example; Multimedia Web sites; Semantics; Social networking (online); Topic modelling; Training
• DOI: 10.1109/SMAP49528.2020.9248465

Automatic Image Annotation (AIA) is the process of assigning tags to digital images without the intervention of humans. Most of the modern automatic image annotation methods are based on the learning by example paradigm. In those methods building the training examples, that is, pairs of images and related tags, is the first critical step. We have shown in our previous studies that hashtags accompanying images in social media and especially the Instagram provide a reach source for creating training sets for AIA. However, we concluded that only 20% of the Instagram hashtags describe the actual content of the image they accompany, thus, a series of filtering steps need to apply in order to identify the appropriate hashtags. In this paper we apply graph based topic modelling on Instagram hashtags in order to predict the subject of the related images and we propose an innovative image retrieval scheme that can be used in the context of Instagram with minimal training requirements.