A Coarse-to-Fine Framework for Cloud Removal in Remote Sensing Image Sequence

• Published in: IEEE transactions on geoscience and remote sensing Vol. 57; no. 8; pp. 5963 - 5974
• Main Authors: Zhang, Yongjun, Wen, Fei, Gao, Zhi, Ling, Xiao
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Change detection; Cloud and shadow removal; Cloud computing; Clouds; Computer simulation; Contamination; Correlation; Detection; Earth; Ground cover; group-sparse; Image classification; Image detection; Image reconstruction; Image sequences; Land cover; low-rank representation; Misalignment; Outliers (statistics); Principal components analysis; Remote sensing; Removal; Restoration; robust principal component analysis (RPCA); Satellites; Sequencing; Shadows; Sparse matrices; Sparsity
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2019.2903594

Clouds and accompanying shadows, which exist in optical remote sensing images with high possibility, can degrade or even completely occlude certain ground-cover information in images, limiting their applicabilities for Earth observation, change detection, or land-cover classification. In this paper, we aim to deal with cloud contamination problems with the objective of generating cloud-removed remote sensing images. Inspired by low-rank representation together with sparsity constraints, we propose a coarse-to-fine framework for cloud removal in the remote sensing image sequence. Leveraging on group-sparsity constraint, we first decompose the observed cloud image sequence of the same area into the low-rank component, group-sparse outliers, and sparse noise, corresponding to cloud-free land-covers, clouds (and accompanying shadows), and noise respectively. Subsequently, a discriminative robust principal component analysis (RPCA) algorithm is utilized to assign aggressive penalizing weights to the initially detected cloud pixels to facilitate cloud removal and scene restoration. Moreover, we incorporate geometrical transformation into a low-rank model to address the misalignment of the image sequence. Significantly superior to conventional cloud-removal methods, neither cloud-free reference image(s) nor additional operations of cloud and shadow detection are required in our method. Extensive experiments on both simulated data and real data demonstrate that our method works effectively, outperforming many state-of-the-art approaches.