Joint Image Clustering and Labeling by Matrix Factorization

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 38; no. 7; pp. 1411 - 1424
• Main Authors: Seunghoon Hong, Jonghyun Choi, Feyereisl, Jan, Bohyung Han, Davis, Larry S.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Clustering algorithms; Feature extraction; Image clustering; image labeling; label feature; Labeling; Linear programming; Matrix decomposition; non-negative matrix factorization with sparsity and orthogonality constraints (SO-NMF); Sparse matrices; Visualization; Image clustering; label feature; non-negative matrix factorization with sparsity and orthogonality constraints (SO-NMF); image labeling
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2015.2487982

We propose a novel algorithm to cluster and annotate a set of input images jointly, where the images are clustered into several discriminative groups and each group is identified with representative labels automatically. For these purposes, each input image is first represented by a distribution of candidate labels based on its similarity to images in a labeled reference image database. A set of these label-based representations are then refined collectively through a non-negative matrix factorization with sparsity and orthogonality constraints; the refined representations are employed to cluster and annotate the input images jointly. The proposed approach demonstrates performance improvements in image clustering over existing techniques, and illustrates competitive image labeling accuracy in both quantitative and qualitative evaluation. In addition, we extend our joint clustering and labeling framework to solving the weakly-supervised image classification problem and obtain promising results.