MPFINet: A Multilevel Parallel Feature Injection Network for Panchromatic and Multispectral Image Fusion

The fusion of a high-spatial-resolution panchromatic (PAN) image and a corresponding low-resolution multispectral (MS) image can yield a high-resolution multispectral (HRMS) image, which is also known as pansharpening. Most previous methods based on convolutional neural networks (CNNs) have achieved...

Full description

Saved in:
Bibliographic Details
Published inRemote sensing (Basel, Switzerland) Vol. 14; no. 23; p. 6118
Main Authors Feng, Yuting, Jin, Xin, Jiang, Qian, Wang, Quanli, Liu, Lin, Yao, Shaowen
Format Journal Article
LanguageEnglish
Published Basel MDPI AG 01.12.2022
Subjects
Artificial neural networks
Computer vision
Deep learning
Feature extraction
image fusion
Image processing
Image reconstruction
Image resolution
Injection
Methods
Multilevel
Neural networks
pansharpening
Quality assessment
Remote sensing
Satellites
self-attention mechanism
Spectral bands
Online AccessGet full text

Cover

More Information
Summary:The fusion of a high-spatial-resolution panchromatic (PAN) image and a corresponding low-resolution multispectral (MS) image can yield a high-resolution multispectral (HRMS) image, which is also known as pansharpening. Most previous methods based on convolutional neural networks (CNNs) have achieved remarkable results. However, information of different scales has not been fully mined and utilized, and still produces spectral and spatial distortion. In this work, we propose a multilevel parallel feature injection network that contains three scale levels and two parallel branches. In the feature extraction branch, a multi-scale perception dynamic convolution dense block is proposed to adaptively extract the spatial and spectral information. Then, the sufficient multilevel features are injected into the image reconstruction branch, and an attention fusion module based on the spectral dimension is designed in order to fuse shallow contextual features and deep semantic features. In the image reconstruction branch, cascaded transformer blocks are employed to capture the similarities among the spectral bands of the MS image. Extensive experiments are conducted on the QuickBird and WorldView-3 datasets to demonstrate that MPFINet achieves significant improvement over several state-of-the-art methods on both spatial and spectral quality assessments.
ISSN:2072-4292
2072-4292
DOI:10.3390/rs14236118