Open-vocabulary Attribute Detection

Vision-language modeling has enabled open-vocabulary tasks where predictions can be queried using any text prompt in a zero-shot manner. Existing open-vocabulary tasks focus on object classes, whereas research on object attributes is limited due to the lack of a reliable attribute-focused evaluation...

Full description

Saved in:
Bibliographic Details
Published in2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 7041 - 7050
Main Authors Bravo, Maria A., Mittal, Sudhanshu, Ging, Simon, Brox, Thomas
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2023
Subjects
Annotations
Benchmark testing
Computational modeling
Computer vision
Datasets and evaluation
Pattern recognition
Predictive models
Vocabulary
Online AccessGet full text

Cover

More Information
Summary:Vision-language modeling has enabled open-vocabulary tasks where predictions can be queried using any text prompt in a zero-shot manner. Existing open-vocabulary tasks focus on object classes, whereas research on object attributes is limited due to the lack of a reliable attribute-focused evaluation benchmark. This paper introduces the Open- Vocabulary Attribute Detection (OVAD) task and the corresponding OVAD benchmark. The objective of the novel task and benchmark is to probe object-level attribute information learned by vision-language models. To this end, we created a clean and densely annotated test set covering 117 attribute classes on the 80 object classes of MS COCO. It includes positive and negative annotations, which enables open-vocabulary evaluation. Overall, the benchmark consists of 1.4 million annotations. For reference, we provide a first baseline method for open-vocabulary attribute detection. Moreover, we demonstrate the benchmark's value by studying the attribute detection performance of several foundation models.
ISSN:2575-7075
DOI:10.1109/CVPR52729.2023.00680