Dark-Aware Network For Fine-Grained Sketch-Based Image Retrieval

• Published in: IEEE signal processing letters Vol. 28; pp. 264 - 268
• Main Authors: Yang, Zhantao, Zhu, Xiaoguang, Qian, Jiuchao, Liu, Peilin
• Format: Journal Article
• Language: English
• Published: New York IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Cloth; Color; Computer vision; Datasets; Edge and dark region network; generative adversarial network; Generative adversarial networks; Generators; Image color analysis; Image edge detection; Image management; Image retrieval; sketch-based image retrieval; Sketches; Testing; Training
• ISSN: 1070-9908; 1558-2361
• DOI: 10.1109/LSP.2020.3043972

Fine-grained sketch-based image retrieval (FG-SBIR) is an emerging topic in high-level computer vision. Among existing methods, edge-only information based ones are convenient to use but incompetent in distinguishing among ambiguous samples. On the other hand, even though the methods using additional color information can significantly improve the retrieval performance, they are not convenient for practical use as the increase of sketch complexity. This paper fills the gap between these two kinds of methods by proposing a novel method, namely dark-aware sketch-based image retrieval (DA-SBIR), which incorporates the dark region information of images into FG-SBIR without a sacrifice of convenience. Our model consists of two branches to process edge structure and dark region, respectively. Specifically, instead of using real color, DA-SBIR only requires some additional free-hand black sketch to represent the dark region. Besides, a Split Generative Adversarial Network is introduced to automatically split a sketch into edge-structure-only and dark-region-only sketches. Moreover, we build a new clothes dataset, SJTU-Cloth, with more ambiguous samples for FG-SBIR. Experimental results on the QMUL-Shoe dataset and our SJTU-Cloth dataset show that our approach achieves consistent improvements over state-of-the-arts. Code and dataset are available at  https://github.com/y2242794082/SBIR.git .