Flickr Distance: A Relationship Measure for Visual Concepts

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 34; no. 5; pp. 863 - 875
• Main Authors: Wu, Lei, Hua, Xian-Sheng, Yu, Nenghai, Ma, Wei-Ying, Li, Shipeng
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.05.2012; IEEE Computer Society
• Subjects: Abstracting and Indexing as Topic; Algorithms; Applied sciences; Artificial Intelligence; Cognition; Computational modeling; Computer science; control theory; systems; Correlation; Data Mining - methods; distance learning; Exact sciences and technology; Feature extraction; Google; Humans; image analysis; Image Processing, Computer-Assisted - methods; Information systems. Data bases; Information theory; Information, signal and communications theory; machine vision; Memory organisation. Data processing; Models, Theoretical; Multimedia; Pattern recognition. Digital image processing. Computational geometry; Semantics; Social Media; Software; Speech and sound recognition and synthesis. Linguistics; Telecommunications and information theory; Visualization; Multimedia; Visual languages; Computer vision; Image processing; Image databank; Electronic document; Cognition; Probability distribution; Text; Human factor; Remote teaching; distance learning; machine vision; Information measure; Image analysis; Semantics; News; Distance measurement; Large scale; Visual information; Computational linguistics; Web site; Artificial intelligence; Distance
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2011.195

This paper proposes the Flickr Distance (FD) to measure the visual correlation between concepts. For each concept, a collection of related images are obtained from the Flickr website. We assume that each concept consists of several states, e.g., different views, different semantics, etc., which are considered as latent topics. Then a latent topic visual language model (LTVLM) is built to capture these states. The Flickr distance between two concepts is defined as the Jensen-Shannon (J-S) divergence between their LTVLM. Differently from traditional conceptual distance measurements, which are based on Web textual documents, FD is based on the visual information. Comparing with the WordNet distance, FD can easily scale up with the increasing size of the conceptual corpus. Comparing with the Google Distance (NGD) and Tag Concurrence Distance (TCD), FD uses the visual information and can properly measure the conceptual relations. We apply FD to multimedia-related tasks and find methods based on FD significantly outperform those based on NGD and TCD. With the FD measurement, we also construct a large-scale visual conceptual network (VCNet) to store the knowledge of conceptual relationship. Experiments show that FD is more coherent to human cognition and it also outperforms text-based distances in real-world applications.