A Coarse-to-Fine Deep Generative Model With Spatial Semantic Attention for High-Resolution Remote Sensing Image Inpainting

Large area array CCD splicing technology and hardware damage lead to the lack of information in high-resolution remote sensing images (HRSIs), which seriously restricts the application of remote sensing images. However, the complex spatial geographical relationship and the serious spatial heterogene...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on geoscience and remote sensing Vol. 60; pp. 1 - 13
Main Authors Du, Yang, He, Jinping, Huang, Qiaolin, Sheng, Qinghong, Tian, Guoliang
Format Journal Article
LanguageEnglish
Published New York IEEE 2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Algorithms
CCD splicing
Charge coupled devices
Coders
deep generative model
Heterogeneity
High resolution
Image resolution
Maintenance engineering
Mathematical models
multilevel loss
Patchiness
Remote sensing
remote sensing image inpainting
Resolution
Satellite imagery
Semantics
Spaceborne remote sensing
Spatial heterogeneity
spatial semantic attention (SSA)
Splicing
Telescopes
Online AccessGet full text

Cover

More Information
Summary:Large area array CCD splicing technology and hardware damage lead to the lack of information in high-resolution remote sensing images (HRSIs), which seriously restricts the application of remote sensing images. However, the complex spatial geographical relationship and the serious spatial heterogeneity will cause texture blur and incomplete structure, especially in the boundary and highly textured regions of HRSI. To deal with the challenge, this article proposes a coarse-to-fine deep generative model with a novel spatial semantic attention (SSA) mechanism. SSA is deployed in the encoder and decoder of the network to ensure the continuity of local features and the relevance of global semantic information and embedded in fine network. In the coarse network, a multilevel loss is proposed to provide more accurate information. Fine network performs more detailed image inpainting operations based on coarse network. Extensive experiments on three datasets, including GF2 satellite images, aerial images, and natural ones, demonstrate that our proposed model has better inpainting effect than the state-of-the-art algorithms.
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2022.3167475