MSAC-Net: 3D Multi-Scale Attention Convolutional Network for Multi-Spectral Imagery Pansharpening

• Published in: Remote sensing (Basel, Switzerland) Vol. 14; no. 12; p. 2761
• Main Authors: Zhang, Erlei, Fu, Yihao, Wang, Jun, Liu, Lu, Yu, Kai, Peng, Jinye
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.06.2022
• Subjects: 3D convolutional; Convolution; deep learning; Design; Image reconstruction; Image resolution; Learning; multi-scale cost; multi-spectral image; Remote sensing; Semantics; Spatial data; Spatial discrimination; Spatial resolution; Supervision
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs14122761

Pansharpening fuses spectral information from the multi-spectral image and spatial information from the panchromatic image, generating super-resolution multi-spectral images with high spatial resolution. In this paper, we proposed a novel 3D multi-scale attention convolutional network (MSAC-Net) based on the typical U-Net framework for multi-spectral imagery pansharpening. MSAC-Net is designed via 3D convolution, and the attention mechanism replaces the skip connection between the contraction and expansion pathways. Multiple pansharpening layers at the expansion pathway are designed to calculate the reconstruction results for preserving multi-scale spatial information. The MSAC-Net performance is verified on the IKONOS and QuickBird satellites’ datasets, proving that MSAC-Net achieves comparable or superior performance to the state-of-the-art methods. Additionally, 2D and 3D convolution are compared, and the influences of the number of convolutions in the convolution block, the weight of multi-scale information, and the network’s depth on the network performance are analyzed.