Bag of spatio-visual words for context inference in scene classification

• Published in: Pattern recognition Vol. 46; no. 3; pp. 1039 - 1053
• Main Authors: Bolovinou, A., Pratikakis, I., Perantonis, S.
• Format: Journal Article
• Language: English
• Published: Kidlington Elsevier Ltd 01.03.2013; Elsevier
• Subjects: Accounting; Applied sciences; Bag of spatio-visual words; Classification; Clustering; Contextual descriptors; Ensembles’ learning; Exact sciences and technology; High dimensional features’ clustering; Image processing; Information, signal and communications theory; Mathematical models; Pattern recognition; Representations; Scene classification; Signal and communications theory; Signal processing; Signal representation. Spectral analysis; Signal, noise; Spatial co-occurrence; Telecommunications and information theory; Visual; Bag of spatio-visual words; Scene classification; High dimensional features’ clustering; Ensembles’ learning; Spatial co-occurrence; Contextual descriptors; High dimensional features' clustering; Automatic classification; Dimensionality; State of the art; Image processing; Occurrence frequency; K means algorithm; Signal classification; Learning; Image analysis; Scene analysis; Ensembles' learning; Image classification
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2012.07.024

In the “bag of visual words (BoVW)” representation each image is represented by an unordered set of visual words. In this paper, a novel approach to encode ordered spatial configurations of visual words in order to add context in the representation is presented. The proposed method introduces a bag of spatio-visual words representation (BoSVW) obtained by clustering of visual words' correlogram ensembles. Specifically, the spherical K-means clustering algorithm is employed accounting for the large dimensionality and the sparsity of the proposed spatio-visual descriptors. Experimental results on four standard datasets show that the proposed method significantly improves a state-of-the-art BoVW model and compares favorably to existing context-based scene classification approaches. ► Reform BoVw representation to include spatio-contextual information. ► Spherical k-means for high-dimentional spatio-visual data clustering. ► Improves a state-of-the-art BoVw model on 4 reference datasets. ► Compares favorably to existing context-based scene classification approaches.