Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening

• Published in: IEEE transactions on geoscience and remote sensing Vol. 59; no. 8; pp. 6995 - 7010
• Main Authors: Deng, Liang-Jian, Vivone, Gemine, Jin, Cheng, Chanussot, Jocelyn
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE); Institute of Electrical and Electronics Engineers
• Subjects: Artificial neural networks; Component substitution (CS); Computer architecture; Convolutional neural networks; Data acquisition; deep convolutional neural network (DCNN); Engineering Sciences; Exploitation; image fusion; Image resolution; Injection; Learning algorithms; Machine learning; Multiresolution analysis; multiresolution analysis (MRA); Neural networks; pansharpening; remote sensing; Resolution; Signal and Image processing; Spatial discrimination; Spatial resolution; Training; pansharpening; multiresolution analysis (MRA); Component substitution (CS); remote sensing; deep convolutional neural network (DCNN); image fusion
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2020.3031366

The fusion of high spatial resolution panchromatic (PAN) data with simultaneously acquired multispectral (MS) data with the lower spatial resolution is a hot topic, which is often called pansharpening. In this article, we exploit the combination of machine learning techniques and fusion schemes introduced to address the pansharpening problem. In particular, deep convolutional neural networks (DCNNs) are proposed to solve this issue. The latter is combined first with the traditional component substitution and multiresolution analysis fusion schemes in order to estimate the nonlinear injection models that rule the combination of the upsampled low-resolution MS image with the extracted details exploiting the two philosophies. Furthermore, inspired by these two approaches, we also developed another DCNN for pansharpening. This is fed by the direct difference between the PAN image and the upsampled low-resolution MS image. Extensive experiments conducted both at reduced and full resolutions demonstrate that this latter convolutional neural network outperforms both the other detail injection-based proposals and several state-of-the-art pansharpening methods.