Deep-seated features histogram: A novel image retrieval method

• Published in: Pattern recognition Vol. 116; p. 107926
• Main Authors: Liu, Guang-Hai, Yang, Jing-Yu
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.08.2021
• Subjects: color perception; deep-seated features histogram; Image retrieval; orientation selection; VGG-16 network; orientation selection; deep-seated features histogram; color perception; Image retrieval; VGG-16 network
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2021.107926

•Low-level features are extracted by simulating the human orientation selection and color perception mechanisms.•Ranking whitening is proposed for extracting deep features via low-level features and reasonably combining them to obtain deep-seated features.•The proposed method is straightforward and reduces the vector dimensionality.•Deep-seated features can describe image contents in terms of colors and edge orientations and identify similar scene styles. Low-level features and deep features each have their own advantages and disadvantages in image representation. However, combining their advantages within a CBIR framework remains challenging. To address this problem, we propose a novel image-retrieval method: the deep-seated features histogram (DSFH). Its main highlights are: 1) Low-level features are extracted by simulating the human orientation selection and color perception mechanisms. This follows the human habit of looking at conspicuous regions and then less-conspicuous ones. 2) A novel method, ranking whitening, is proposed for extracting deep features via low-level features and combining them to obtain deep-seated features. 3) The proposed method is straightforward and reduces the vector dimensionality of the FC7 layer of a pre-trained VGG-16 network, and significantly improves image-retrieval precision. Comparative experiments demonstrate that the proposed method outperforms several state-of-the-art methods, including low-level feature-based, deep feature-based, and fused feature-based methods, in terms of precision/recall, area under the precision/recall curve metrics, and mean average precision. The proposed method provides efficient CBIR performance and not only has the power to discriminate low-level features, including color, texture, and shape, but can also match scenes of similar style.