Oboyob: A sequential-semantic Bengali image captioning engine

• Published in: Journal of intelligent & fuzzy systems Vol. 37; no. 6; pp. 7427 - 7439
• Main Authors: Deb, Tonmoay, Ali, Mohammad Zariff Ahsham, Bhowmik, Sanchita, Firoze, Adnan, Ahmed, Syed Shahir, Tahmeed, Muhammad Abeer, Rahman, N.S.M. Rezaur, Rahman, Rashedur M.
• Format: Journal Article
• Language: English
• Published: Amsterdam IOS Press BV 01.01.2019
• Subjects: Computer vision; Datasets; Embedding; Natural language processing; Training
• ISSN: 1064-1246; 1875-8967
• DOI: 10.3233/JIFS-179351

Understanding the context with generation of textual description from an input image is an active and challenging research topic in computer vision and natural language processing. However, in the case of Bengali language, the problem is still unexplored. In this paper, we address a standard approach for Bengali image caption generation though subsampling the machine translated dataset. Later, we use several pre-processing techniques with the state-of-the-art CNN-LSTM architecture-based models. The experiment is conducted on standard Flickr-8K dataset, along with several modifications applied to adapt with the Bengali language. The training caption subsampled dataset is computed for both Bengali and English languages for further experiments with 16 distinct models developed in the entire training process. The trained models for both languages are analyzed with respect to several caption evaluation metrics. Further, we establish a baseline performance in Bengali image captioning defining the limitation of current word embedding approaches compared to internal local embedding.