Cross-modality Features Fusion for Synthetic Aperture Radar Image Segmentation

• Published in: IEEE transactions on geoscience and remote sensing Vol. 61; p. 1
• Main Authors: Gao, Fei, Huang, Heqing, Yue, Zhenyu, Li, Dongyu, Ge, Shuzhi Sam, Lee, Tong Heng, Zhou, Huiyu
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Coders; conditional random field; Conditional random fields; Context modeling; Convolutional neural networks; cross-modality features; Data mining; Feature extraction; fully convolutional network; Image enhancement; Image processing; Image segmentation; Pixels; Radar; Radar imaging; Radar polarimetry; SAR (radar); Synthetic aperture radar; Transformers
• DOI: 10.1109/TGRS.2023.3307825
• ISSN: 0196-2892

Synthetic Aperture Radar (SAR) image segmentation stands as a formidable research frontier within the domain of SAR image interpretation. The fully convolutional network (FCN) methods have recently brought remarkable improvements in SAR image segmentation. Nevertheless, these methods do not utilize the peculiarities of SAR images, leading to suboptimal segmentation accuracy. To address this issue, we rethink SAR image segmentation in terms of sequential information of transformers and cross-modal features. We first discuss the peculiarities of SAR images and extract the mean and texture features utilized as auxiliary features. The extraction of auxiliary features helps unearth the distinctive information in the SAR images. Afterward, a feature-enhanced FCN with the transformer encoder structure, termed FE-FCN, which can be extracted to context-level and pixel-level features. In FE-FCN, the features of a single-mode encoder are aligned and inserted into the model to explore the potential correspondence between modes. We also employ long skip connections to share each modality's distinguishing and particular features. Finally, we present the connection-enhanced conditional random field (CE-CRF) to capture the connection information of the image pixels. Since the CE-CRF utilizes the auxiliary features to enhance the reliability of the connection information, the segmentation results of FE-FCN are further optimized. Comparative experiments conducted on the Fangchenggang (FCG), Pucheng (PC), and Gaofen (GF) SAR datasets. Our method demonstrates superior segmentation accuracy compared to other conventional image segmentation methods, as confirmed by the experimental results.