Convolutional Analysis Sparse Coding for Multimodal Image Super-Resolution

• Published in: IEEE sensors letters Vol. 8; no. 6; pp. 1 - 4
• Main Authors: Kumar, Kriti, Majumdar, Angshul, Kumar, A. Anil, Chandra, M. Girish
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.06.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Benchmark testing; Convolutional codes; convolutional sparse coding; convolutional transform learning (CTL); Image coding; Image enhancement; Image reconstruction; Image resolution; Inverse problems; joint optimization; Learning; multimodal image super-resolution (MISR); Performance enhancement; Sensor applications; Superresolution; Task analysis; Transforms
• ISSN: 2475-1472; 2475-1472
• DOI: 10.1109/LSENS.2024.3403179

With multimodal imaging systems in place, recent research focus has been directed toward exploiting the knowledge from different imaging modalities to solve inverse problems, one example being image super-resolution, considered in this letter. The goal of multimodal image super-resolution (MISR) is to enhance the resolution of low-resolution (LR) images from the target modality by taking guidance from high-resolution (HR) images from a different modality. Conventional methods for MISR employing convolutional neural networks typically adopt an encoder-decoder architecture that often demands massive data for optimal reconstruction and tends to overfit in data-limited scenarios. The proposed work presents a convolutional analysis sparse coding-based method, employing convolutional transforms that eliminate the need for learning the decoder network and learns unique filters. This reduces the number of trainable parameters, making it suitable for learning with limited data. A joint optimization formulation is presented that learns dedicated convolutional transforms for both the LR images of target modality and the HR images of guidance modality, and a fusing (combining) transform that combines the respective transform features to reconstruct the HR images of target modality. Unlike the dictionary-based synthesis sparse coding methods for MISR, the proposed approach offers enhanced performance with reduced complexity due to some of the inherent advantages of transform learning. The effectiveness of the proposed method is validated using RGB-Near Infrared and RGB-Multispectral datasets. Experimental findings demonstrate enhanced reconstruction performance achieved by the proposed method, compared with the state-of-the-art techniques. Moreover, the proposed method demonstrates improved robustness to noise compared with other sparse coding-based methods.