Attribute-Based Classification for Zero-Shot Visual Object Categorization

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 36; no. 3; pp. 453 - 465
• Main Authors: Lampert, Christoph H., Nickisch, Hannes, Harmeling, Stefan
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.03.2014; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Animals; Applied sciences; Artificial intelligence; Classification; Classification - methods; Computer science; control theory; systems; Computer vision; Databases, Factual; Exact sciences and technology; Image Processing, Computer-Assisted - methods; Learning; Marine animals; Models, Statistical; Object recognition; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Probabilistic logic; Programming languages; Semantics; Software; Support Vector Machine; Tasks; Training; Vectors; vision and scene understanding; Computer vision; Zero-shot learning; Image interpretation; Image databank; Pattern recognition; Object recognition; Annotation; Semantics; Scene analysis; Animal; vision and scene understanding; Object oriented; Collection; Categorization; Image classification
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2013.140

We study the problem of object recognition for categories for which we have no training examples, a task also called zero--data or zero-shot learning. This situation has hardly been studied in computer vision research, even though it occurs frequently; the world contains tens of thousands of different object classes, and image collections have been formed and suitably annotated for only a few of them. To tackle the problem, we introduce attribute-based classification: Objects are identified based on a high-level description that is phrased in terms of semantic attributes, such as the object's color or shape. Because the identification of each such property transcends the specific learning task at hand, the attribute classifiers can be prelearned independently, for example, from existing image data sets unrelated to the current task. Afterward, new classes can be detected based on their attribute representation, without the need for a new training phase. In this paper, we also introduce a new data set, Animals with Attributes, of over 30,000 images of 50 animal classes, annotated with 85 semantic attributes. Extensive experiments on this and two more data sets show that attribute-based classification indeed is able to categorize images without access to any training images of the target classes.