Event-Aware Retrospective Learning for Knowledge-Based Image Captioning

• Published in: IEEE transactions on multimedia Vol. 26; pp. 4898 - 4911
• Main Authors: Liu, An-An, Zhai, Yingchen, Xu, Ning, Tian, Hongshuo, Nie, Weizhi, Zhang, Yongdong
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Ablation; Algorithms; Bioinformatics; Correlation; Datasets; external knowledge; Genomics; Image captioning; Knowledge; Knowledge based systems; Knowledge engineering; Knowledge representation; Learning; Retrieval; retrospective learning; Semantics; Sentences; visual facts; Visualization

External knowledge has been widely applied in image captioning tasks to enrich the generated sentences. However, existing methods retrieve knowledge by considering only semantic relevance while ignoring whether they are useful for captioning. For example, when querying "person" in external...