A Cross-Modality Feature Transfer Method for Target Detection in SAR Images

• Published in: IEEE transactions on geoscience and remote sensing Vol. 61; pp. 1 - 15
• Main Authors: He, Jiayue, Su, Nan, Xu, Congan, Liao, Yanping, Yan, Yiming, Zhao, Chunhui, Hou, Wei, Feng, Shou
• Format: Journal Article
• Language: English
• Published: New York IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Alignment; Color imagery; Cross-modality feature transfer (CMFT); Data mining; Datasets; deep learning; Detection; Detectors; Feature extraction; Marine vehicles; Modules; Optimization; Radar detection; Radar imaging; Radar polarimetry; Representations; SAR (radar); Ships; Synthetic aperture radar; synthetic aperture radar (SAR) ship detection; Target detection; Task analysis
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2023.3300536

Synthetic aperture radar (SAR) ship detection methods have achieved remarkable progress in recent years. However, unlike RGB images, the characteristics of SAR imaging will result in non-intuitive feature representations. Furthermore, due to the insufficient data of SAR images, existing methods relying on plenty of labeled SAR images may be hard to achieve promising performance. To address the aforementioned issues, a cross-modality feature transfer (CMFT) method is proposed in this article, which enhances feature representations in the SAR modality by transferring rich knowledge in the RGB modality. First, we propose a multilevel modality alignment network (MMAN), which encourages the model to effectively learn modality-invariant features and alleviate the large cross-modality discrepancies by aligning features from multilevels (scene level, local level, global level, and instance level). Second, to address the underperformance of samples with non-intuitive features in the modality alignment, we introduce a hard-sample supervision module (HSM) in the stage of feature extraction, which can thoroughly exploit the feature of hard-to-align samples by giving more optimization energy for them. Third, to enhance the discriminability of instance-level features, a feature complementary module (FCM) is customized to fully explore the potential complementary clues between instance-level features and context information for the instance-level feature alignment. Extensive experimental results demonstrate that the CMFT outperforms the state-of-the-art detectors. Compared to the baseline model, CMFT improves the accuracy by 3.1% mean average precision (mAP) on the SSDD dataset and 3.4% mAP on the HRSID dataset, demonstrating its superior SAR ship detection performance.