Aesthetics-Guided Graph Clustering With Absent Modalities Imputation

• Published in: IEEE transactions on image processing Vol. 28; no. 7; pp. 3462 - 3476
• Main Authors: Zhang, Luming, Yao, Yiyang, Liu, Zhenguang, Shao, Ling
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: aesthetic community; Aesthetics; Algorithms; Clustering; Clustering algorithms; Communities; Computational modeling; Data mining; Divergence; Feature extraction; Flickr; gaze behavior; graph mining; Graph theory; Human behavior; Image detection; Internet; Machine learning; multimodal; Outliers (statistics); partially-supervised; Semantics; Training; Video data; Visualization
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2019.2897940

Accurately clustering Internet-scale Internet users into multiple communities according to their aesthetic styles is a useful technique in image modeling and data mining. In this paper, we present a novel partially supervised model which seeks a sparse representation to capture photo aesthetics. It optimally fuzes multi-channel features, i.e., human gaze behavior, quality scores, and semantic tags, each of which could be absent. Afterward, by leveraging the KL-divergence to distinguish the aesthetic distributions between photo sets, a large-scale graph is constructed to describe the aesthetic correlations between users. Finally, a dense subgraph mining algorithm which intrinsically supports outliers (i.e., unique users not belong to any community) is adopted to detect aesthetic communities. The comprehensive experimental results on a million-scale image set grabbed from Flickr have demonstrated the superiority of our method. As a byproduct, the discovered aesthetic communities can enhance photo retargeting and video summarization substantially.