MSISR-STF: Spatiotemporal Fusion via Multilevel Single-Image Super-Resolution

• Published in: Remote sensing (Basel, Switzerland) Vol. 15; no. 24; p. 5675
• Main Authors: Zheng, Xiongwei, Feng, Ruyi, Fan, Junqing, Han, Wei, Yu, Shengnan, Chen, Jia
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.12.2023
• Subjects: Algorithms; Artificial intelligence; Comparative analysis; Complementarity; Computer vision; data collection; Data fusion; Data integration; Graph neural networks; graphs; IGNN; image analysis; Image processing; Image resolution; Interpolation; Methods; Multilevel; Neural networks; Noise control; Noise generation; Remote sensing; Satellites; SISR; Spatial data; Spatial discrimination; Spatial resolution; spatial variation; Spatiotemporal data; spatiotemporal image fusion; Technology application; Temporal resolution; Thin plates; TPS; China
• DOI: 10.3390/rs15245675
• ISSN: 2072-4292

Due to technological limitations and budget constraints, spatiotemporal image fusion uses the complementarity of high temporal–low spatial resolution (HTLS) and high spatial–low temporal resolution (HSLT) data to obtain high temporal and spatial resolution (HTHS) fusion data, which can effectively satisfy the demand for HTHS data. However, some existing spatiotemporal image fusion models ignore the large difference in spatial resolution, which yields worse results for spatial information under the same conditions. Based on the flexible spatiotemporal data fusion (FSDAF) framework, this paper proposes a multilevel single-image super-resolution (SISR) method to solve this issue under the large difference in spatial resolution. The following are the advantages of the proposed method. First, multilevel super-resolution (SR) can effectively avoid the limitation of a single SR method for a large spatial resolution difference. In addition, the issue of noise accumulation caused by multilevel SR can be alleviated by learning-based SR (the cross-scale internal graph neural network (IGNN)) and then interpolation-based SR (the thin plate spline (TPS)). Finally, we add the reference information to the super-resolution, which can effectively control the noise generation. This method has been subjected to comprehensive experimentation using two authentic datasets, affirming that our proposed method surpasses the current state-of-the-art spatiotemporal image fusion methodologies in terms of performance and effectiveness.