Fast unsupervised consistent and modality-specific hashing for multimedia retrieval

• Published in: Neural computing & applications Vol. 35; no. 8; pp. 6207 - 6223
• Main Authors: Yang, Zhan, Deng, Xiyin, Long, Jun
• Format: Journal Article
• Language: English
• Published: London Springer London 01.03.2023; Springer Nature B.V
• Subjects: Annotations; Artificial Intelligence; Binary codes; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Data storage; Decomposition; Image Processing and Computer Vision; Information retrieval; Learning; Modules; Multimedia; Optimization; Original Article; Probability and Statistics in Computer Science; Representations; Retrieval performance measures; Semantics; Supervision; Hashing; Discrete optimization; Cross-modal retrieval
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-022-08008-4

Hashing is an effective technique to solve large-scale data storage problem and achieve efficient retrieval, and it is also a core technology to promote the intelligent development of the new infrastructure construction. In most practical situations, label information is unavailable, and creating manual annotations is a time-consuming and laborious process. Therefore, unsupervised cross-modal hashing technique has received extensive attention from the information retrieval community due to its fast retrieval speed and feasibility. However, the capabilities of existing unsupervised cross-modal hashing methods are not sufficient to comprehensively describe the complex relations among different modalities, such as the balance of complementary and consistency between different modalities. In this article, we propose a new-type of unsupervised cross-modal hashing method called F ast U nsupervised C onsistent and M odality- S pecific H ashing ( FUCMSH ). Specifically, FUCMSH  consists of two main modules, i.e., shared matrix factorization module (SMFM) and individual auto-encoding module (IAEM). In the SMFM, FUCMSH  dynamically assigns weights to different modalities to adaptively balance the contribution of different modalities. By doing so, the information completeness of the shared consistent representation can be guaranteed. In the IAEM, FUCMSH  learns individual modality-specific latent representations of different modalities through modality-specific linear autoencoders. Moreover, FUCMSH  makes use of the transfer learning to link the relationships between different individual modality-specific latent representations. Combined with the SMFM and the IAEM, the discriminative capability of the generated binary codes can be significantly improved. The relatively extensive experimental results manifest the superiority of the proposed FUCMSH .