Tagging Like Humans: Diverse and Distinct Image Annotation

In this work we propose a new automatic image annotation model, dubbed diverse and distinct image annotation (D2IA). The generative model D2IA is inspired by the ensemble of human annotations, which create semantically relevant, yet distinct and diverse tags. In D2IA, we generate a relevant and dist...

Full description

Saved in:
Bibliographic Details
Published in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 7967 - 7975
Main Authors Wu, Baoyuan, Chen, Weidong, Sun, Peng, Liu, Wei, Ghanem, Bernard, Lyu, Siwei
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2018
Subjects
Gallium nitride
Generators
Image annotation
Redundancy
Semantics
Task analysis
Training
Online AccessGet full text
ISSN1063-6919
DOI10.1109/CVPR.2018.00831

Cover

More Information
Summary:In this work we propose a new automatic image annotation model, dubbed diverse and distinct image annotation (D2IA). The generative model D2IA is inspired by the ensemble of human annotations, which create semantically relevant, yet distinct and diverse tags. In D2IA, we generate a relevant and distinct tag subset, in which the tags are relevant to the image contents and semantically distinct to each other, using sequential sampling from a determinantal point process (DPP) model. Multiple such tag subsets that cover diverse semantic aspects or diverse semantic levels of the image contents are generated by randomly perturbing the DPP sampling process. We leverage a generative adversarial network (GAN) model to train D2IA. Extensive experiments including quantitative and qualitative comparisons, as well as human subject studies, on two benchmark datasets demonstrate that the proposed model can produce more diverse and distinct tags than the state-of-the-arts.
ISSN:1063-6919
DOI:10.1109/CVPR.2018.00831