Fusion of Deep Learning and Compressed Domain Features for Content-Based Image Retrieval

• Published in: IEEE transactions on image processing Vol. 26; no. 12; pp. 5706 - 5717
• Main Authors: Liu, Peizhong, Guo, Jing-Ming, Wu, Chi-Yi, Cai, Danlin
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.12.2017
• Subjects: block truncation coding; Computational complexity; Content-based image retrieval; convolutional-neural network; deep learning; Feature extraction; halftoning; Histograms; Image coding; Image color analysis; Image retrieval; Machine learning
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2017.2736343

This paper presents an effective image retrieval method by combining high-level features from convolutional neural network (CNN) model and low-level features from dot-diffused block truncation coding (DDBTC). The low-level features, e.g., texture and color, are constructed by vector quantization -indexed histogram from DDBTC bitmap, maximum, and minimum quantizers. Conversely, high-level features from CNN can effectively capture human perception. With the fusion of the DDBTC and CNN features, the extended deep learning two-layer codebook features is generated using the proposed two-layer codebook, dimension reduction, and similarity reweighting to improve the overall retrieval rate. Two metrics, average precision rate and average recall rate (ARR), are employed to examine various data sets. As documented in the experimental results, the proposed schemes can achieve superior performance compared with the state-of-the-art methods with either low-or high-level features in terms of the retrieval rate. Thus, it can be a strong candidate for various image retrieval related applications.