Label Consistent Matrix Factorization Hashing for Large-Scale Cross-Modal Similarity Search

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 41; no. 10; pp. 2466 - 2479
• Main Authors: Wang, Di, Gao, Xinbo, Wang, Xiumei, He, Lihuo
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Analogies; Binary codes; Correlation; cross-modal; Datasets; Factorization; Hashing; Image reconstruction; Labels; Multimedia; multimodal; Representations; Searching; Semantics; Similarity; similarity search; Sparse matrices; supervised; Training; Transforms
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2018.2861000

Multimodal hashing has attracted much interest for cross-modal similarity search on large-scale multimedia data sets because of its efficiency and effectiveness. Recently, supervised multimodal hashing, which tries to preserve the semantic information obtained from the labels of training data, has received considerable attention for its higher search accuracy compared with unsupervised multimodal hashing. Although these algorithms are promising, they are mainly designed to preserve pairwise similarities. When semantic labels of training data are given, the algorithms often transform the labels into pairwise similarities, which gives rise to the following problems: (1) constructing pairwise similarity matrix requires enormous storage space and a large amount of calculation, making these methods unscalable to large-scale data sets; (2) transforming labels into pairwise similarities loses the category information of the training data. Therefore, these methods do not enable the hash codes to preserve the discriminative information reflected by labels and, hence, the retrieval accuracies of these methods are affected. To address these challenges, this paper introduces a simple yet effective supervised multimodal hashing method, called label consistent matrix factorization hashing (LCMFH), which focuses on directly utilizing semantic labels to guide the hashing learning procedure. Considering that relevant data from different modalities have semantic correlations, LCMFH transforms heterogeneous data into latent semantic spaces in which multimodal data from the same category share the same representation. Therefore, hash codes quantified by the obtained representations are consistent with the semantic labels of the original data and, thus, can have more discriminative power for cross-modal similarity search tasks. Thorough experiments on standard databases show that the proposed algorithm outperforms several state-of-the-art methods.