LANet: Local Attention Embedding to Improve the Semantic Segmentation of Remote Sensing Images

• Published in: IEEE transactions on geoscience and remote sensing Vol. 59; no. 1; pp. 426 - 435
• Main Authors: Ding, Lei, Tang, Hao, Bruzzone, Lorenzo
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Convolutional neural network (CNN); Convolutional neural networks; Correlation; Decoding; deep learning; Embedding; Feature extraction; Geographical distribution; Image classification; Image processing; Image segmentation; Localization; Modules; Neural networks; Remote sensing; Representations; Semantic segmentation; Semantics; Spatial discrimination; Spatial distribution
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2020.2994150

The trade-off between feature representation power and spatial localization accuracy is crucial for the dense classification/semantic segmentation of remote sensing images (RSIs). High-level features extracted from the late layers of a neural network are rich in semantic information, yet have blurred spatial details; low-level features extracted from the early layers of a network contain more pixel-level information but are isolated and noisy. It is therefore difficult to bridge the gap between high- and low-level features due to their difference in terms of physical information content and spatial distribution. In this article, we contribute to solve this problem by enhancing the feature representation in two ways. On the one hand, a patch attention module (PAM) is proposed to enhance the embedding of context information based on a patchwise calculation of local attention. On the other hand, an attention embedding module (AEM) is proposed to enrich the semantic information of low-level features by embedding local focus from high-level features. Both proposed modules are lightweight and can be applied to process the extracted features of convolutional neural networks (CNNs). Experiments show that, by integrating the proposed modules into a baseline fully convolutional network (FCN), the resulting local attention network (LANet) greatly improves the performance over the baseline and outperforms other attention-based methods on two RSI data sets.