ECEA: Extensible Co-Existing Attention for Few-Shot Object Detection

• Published in: IEEE transactions on image processing Vol. 33; pp. 5564 - 5576
• Main Authors: Xin, Zhimeng, Wu, Tianxu, Chen, Shiming, Zou, Yixiong, Shao, Ling, You, Xinge
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2024
• Subjects: Adaptation models; co-existing regions; Detectors; extensible attention; Feature extraction; Few-shot object detection; Object detection; Semantics; Task analysis; Training
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2024.3411771

Few-shot object detection (FSOD) identifies objects from extremely few annotated samples. Most existing FSOD methods, recently, apply the two-stage learning paradigm, which transfers the knowledge learned from abundant base classes to assist the few-shot detectors by learning the global features. However, such existing FSOD approaches seldom consider the localization of objects from local to global. Limited by the scarce training data in FSOD, the training samples of novel classes typically capture part of objects, resulting in such FSOD methods being unable to detect the completely unseen object during testing. To tackle this problem, we propose an Extensible Co-Existing Attention (ECEA) module to enable the model to infer the global object according to the local parts. Specifically, we first devise an extensible attention mechanism that starts with a local region and extends attention to co-existing regions that are similar and adjacent to the given local region. We then implement the extensible attention mechanism in different feature scales to progressively discover the full object in various receptive fields. In the training process, the model learns the extensible ability on the base stage with abundant samples and transfers it to the novel stage of continuous extensible learning, which can assist the few-shot model to quickly adapt in extending local regions to co-existing regions. Extensive experiments on the PASCAL VOC and COCO datasets show that our ECEA module can assist the few-shot detector to completely predict the object despite some regions failing to appear in the training samples and achieve the new state-of-the-art compared with existing FSOD methods. Code is released at https://github.com/zhimengXin/ECEA .