SE-FewDet: Semantic-Enhanced Feature Generation and Prediction Refinement for Few-Shot Object Detection

• Published in: 2024 International Joint Conference on Neural Networks (IJCNN) pp. 1 - 8
• Main Authors: Zhang, Yineng, Liu, Sheng, Feng, Yuan, Pan, Songqi, Tian, Xiaopeng, Yang, Jiantao
• Format: Conference Proceeding
• Language: English
• Published: IEEE 30.06.2024
• Subjects: Feature extraction; feature generation; Few-shot learning; Filters; Generators; Neural networks; Object detection; Semantics; VAE; Visualization
• ISSN: 2161-4407
• DOI: 10.1109/IJCNN60899.2024.10650252

Few-Shot Object Detection (FSOD) entails learning from few examples. Due to the lack of data diversity, feature generation emerges as an effective method to improve performance. However, the process of generating diverse features for the novel classes, which introduces excessive intra-class variations of the base classes, resulting in blurring the boundaries between the novel and the base classes. To ensure the diversity and boundary clarity of the generated features, our SE-FewDet explores a new structure called SemVAE to integrate semantic and visual information. This structure allows the generator to strengthen the class-centred representation through cross-modal constraints, thus clarifying the boundaries of different classes while ensuring the enhancement of data diversity. Additionally, our SE-FewDet includes a semantic-enhanced prediction refinement module that accurately filters out potential false positives caused by bounding box offsets, ensuring that only the most reliable detections remain. We evaluate our approach on the PASCAL VOC and MS COCO datasets. With these improvements, SE-FewDet significantly improves detection performance on new classes compared to the baseline (VFA).