FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network

• Published in: IEEE transactions on multimedia Vol. 25; pp. 1 - 13
• Main Authors: Liu, Huafeng, Peng, Pai, Chen, Tao, Wang, Qiong, Yao, Yazhou, Hua, Xian-Sheng
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Context; Correlation; Few-shot learning; Few-shot semantic segmentation; Image reconstruction; Learning visual correspondence; Matching; Modules; Prototypes; Semantic segmentation; Semantics; Task analysis; Training; Visualization
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2023.3238521

Few-shot semantic segmentation is the task of learning to locate each pixel of the novel class in the query image with only a few annotated support images. The current correlation-based methods construct pair-wise feature correlations to establish the many-to-many matching because the typical prototype-based approaches cannot learn fine-grained correspondence relations. However, the existing methods still suffer from the noise contained in naive correlations and the lack of context semantic information in correlations. To alleviate these problems mentioned above, we propose a Feature-Enhanced Context-Aware Network (FECANet). Specifically, a feature enhancement module is proposed to suppress the matching noise caused by inter-class local similarity and enhance the intra-class relevance in the naive correlation. In addition, we propose a novel correlation reconstruction module that encodes extra correspondence relations between foreground and background and multi-scale context semantic features, significantly boosting the encoder to capture a reliable matching pattern. Experiments on PASCAL-<inline-formula><tex-math notation="LaTeX">5^{i}</tex-math></inline-formula> and COCO-<inline-formula><tex-math notation="LaTeX">20^{i}</tex-math></inline-formula> datasets demonstrate that our proposed FECANet leads to remarkable improvement compared to previous state-of-the-arts, demonstrating its effectiveness. The source codes and models have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/FECANET .