Boosting Few-Shot Fine-Grained Recognition With Background Suppression and Foreground Alignment

• Published in: IEEE transactions on circuits and systems for video technology Vol. 33; no. 8; pp. 3947 - 3961
• Main Authors: Zha, Zican, Tang, Hao, Sun, Yunlian, Tang, Jinhui
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Alignment; Annotations; background suppression; Birds; Feature extraction; Feature maps; Few-shot learning; fine-grained recognition; foreground alignment; Measurement; Misalignment; Modules; Recognition; Similarity; Sun; Task analysis; Training
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2023.3236636

Few-shot fine-grained recognition (FS-FGR) aims to recognize novel fine-grained categories with the help of limited available samples. Undoubtedly, this task inherits the main challenges from both few-shot learning and fine-grained recognition. First, the lack of labeled samples makes the learned model easy to overfit. Second, it also suffers from high intra-class variance and low inter-class differences in the datasets. To address this challenging task, we propose a two-stage background suppression and foreground alignment framework, which is composed of a background activation suppression (BAS) module, a foreground object alignment (FOA) module, and a local-to-local (L2L) similarity metric. Specifically, the BAS is introduced to generate a foreground mask for localization to weaken background disturbance and enhance dominative foreground objects. The FOA then reconstructs the feature map of each support sample according to its correction to the query ones, which addresses the problem of misalignment between support-query image pairs. To enable the proposed method to have the ability to capture subtle differences in confused samples, we present a novel L2L similarity metric to further measure the local similarity between a pair of aligned spatial features in the embedding space. What's more, considering that background interference brings poor robustness, we infer the pairwise similarity of feature maps using both the raw image and the refined image. Extensive experiments conducted on multiple popular fine-grained benchmarks demonstrate that our method outperforms the existing state of the art by a large margin. The source codes are available at: https://github.com/CSer-Tang-hao/BSFA-FSFG .