SI-SA GAN: A Generative Adversarial Network Combined with Spatial Information and Self-Attention for Removing Thin Cloud in Optical Remote Sensing Images

• Published in: IEEE access Vol. 10; p. 1
• Main Authors: Liu, Juntao, Hou, Weimin, Luo, Xin, Su, Jia, Hou, Yanli, Wang, Zhenzhou
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Agriculture; Clouds; Data acquisition; Data integration; Feature extraction; GAN; Generative adversarial networks; Image acquisition; Image enhancement; Image quality; Multisensor fusion; Neural networks; Occlusion; Optical sensors; Performance indices; Remote monitoring; Remote sensing; Satellite imagery; self-attention; Spatial data; spatial multi-directional perception; Spatial resolution; thin cloud removal
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2022.3213354

In agricultural remote sensing monitoring, the quality of optical remote sensing image data acquisition is often affected by climate, and the acquired satellite imagery results usually contain cloud information, which can lead to a lack of ground data information. Unlike thick clouds, the semi-transparent nature of thin clouds prevents thin clouds from completely obscuring the ground scene. In order to remove thin clouds in the cultivated land and restore the real ground information as much as possible, we proposed a cloud removal method of spatial information fusion self-attention generative adversarial network (SI-SA GAN) based on multi-directional perceptual attention and self-attention mechanism. The proposed method identifies and focuses on cloud regions by using the spatial attention, channel attention, and self-attention mechanism, which can enhance image information. The modules of the discriminator utilize residual networks and self-attention non-local neural networks to guide the output of image information. The generative adversarial network (GAN) is applied to remove clouds and restore the corresponding irregular occlusion area according to the depth characteristics of the input information, and a gradient penalty is applied to improve the robustness of the generative network. In this paper, we compared the evaluation indexes of other advanced models, the qualitative and quantitative results of Sentinel-2A datasets and public RICE datasets confirmed that the proposed method could enhance the image quality effectively after cloud removal, and the model has excellent thin cloud removal performance with small-scale training data.