Task-Adaptive Attention for Image Captioning

• Published in: IEEE transactions on circuits and systems for video technology Vol. 32; no. 1; pp. 43 - 51
• Main Authors: Yan, Chenggang, Hao, Yiming, Li, Liang, Yin, Jian, Liu, Anan, Mao, Zhendong, Chen, Zhenyu, Gao, Xingyu
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; attention mechanism; Computational modeling; Decoding; Feature extraction; Feeds; Image captioning; Modules; Regularization; Task analysis; transformer; Visualization

Attention mechanisms are now widely used in image captioning models. However, most attention models only focus on visual features. When generating syntax related words, little visual information is needed. In this case, these attention models could mislead the word generation. In this paper, we prop...