Spoken Attributes: Mixing Binary and Relative Attributes to Say the Right Thing

• Published in: 2013 IEEE International Conference on Computer Vision pp. 2160 - 2167
• Main Authors: Sadovnik, Amir, Gallagher, Andrew, Parikh, Devi, Tsuhan Chen
• Format: Conference Proceeding Journal Article
• Language: English
• Published: IEEE 01.12.2013
• Subjects: attributes; Classifiers; Computer vision; Conferences; Educational institutions; Face; Feature extraction; Glass; Human; Learning; Mathematical models; Object recognition; relative attributes; Support vector machines; Trains; Vectors; visual attributes
• ISSN: 1550-5499; 2380-7504
• DOI: 10.1109/ICCV.2013.268

In recent years, there has been a great deal of progress in describing objects with attributes. Attributes have proven useful for object recognition, image search, face verification, image description, and zero-shot learning. Typically, attributes are either binary or relative: they describe either the presence or absence of a descriptive characteristic, or the relative magnitude of the characteristic when comparing two exemplars. However, prior work fails to model the actual way in which humans use these attributes in descriptive statements of images. Specifically, it does not address the important interactions between the binary and relative aspects of an attribute. In this work we propose a spoken attribute classifier which models a more natural way of using an attribute in a description. For each attribute we train a classifier which captures the specific way this attribute should be used. We show that as a result of using this model, we produce descriptions about images of people that are more natural and specific than past systems.