Rich Embedding Features for One-Shot Semantic Segmentation

• Published in: IEEE transaction on neural networks and learning systems Vol. 33; no. 11; pp. 6484 - 6493
• Main Authors: Zhang, Xiaolin, Wei, Yunchao, Li, Zhao, Yan, Chenggang, Yang, Yi
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.11.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Computer applications; Context; Deep learning; Embedding; Feature extraction; few shot segmentation; Image segmentation; object segmentation; Pattern analysis; Prototypes; Pulse modulation; Semantic segmentation; Semantics; Siamese network; Support vector machines; Task analysis; Visual discrimination learning
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3081693

One-shot semantic segmentation poses the challenging task of segmenting object regions from unseen categories with only one annotated example as guidance. Thus, how to effectively construct robust feature representations from the guidance image is crucial to the success of one-shot semantic segmentation. To this end, we propose in this article a simple, yet effective approach named rich embedding features (REFs). Given a reference image accompanied with its annotated mask, our REF constructs rich embedding features of the support object from three perspectives: 1) global embedding to capture the general characteristics; 2) peak embedding to capture the most discriminative information; 3) adaptive embedding to capture the internal long-range dependencies. By combining these informative features, we can easily harvest sufficient and rich guidance even from a single reference image. In addition to REF, we further propose a simple depth-priority context module to obtain useful contextual cues from the query image. This successfully raises the performance of one-shot semantic segmentation to a new level. We conduct experiments on pattern analysis, statical modeling and computational learning (Pascal) visual object classes (VOC) 2012 and common object in context (COCO) to demonstrate the effectiveness of our approach.