CoT-MISR:Marrying convolution and transformer for multi-image super-resolution

• Published in: Multimedia tools and applications Vol. 83; no. 31; pp. 76891 - 76903
• Main Authors: Song, Qing, Xiu, Mingming, Nie, Yang, Hu, Mengjie, Liu, Chun
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2024; Springer Nature B.V
• Subjects: Computer Communication Networks; Computer Science; Convolution; Data Structures and Information Theory; Datasets; Deep learning; High resolution; Image enhancement; Image quality; Image resolution; Image restoration; Multimedia Information Systems; Networks; Special Purpose and Application-Based Systems; Track 6: Computer Vision for Multimedia Applications; Transformers; Convolutional; Transformer; Multi-Image Super-resolution(MISR); Deep Learning
• ISSN: 1573-7721; 1380-7501; 1573-7721
• DOI: 10.1007/s11042-024-18591-4

Image super-resolution, a technique for image restoration, has been the subject of extensive research. The challenge lies in converting a low-resolution image to recover its high-resolution information, a problem that researchers have been persistently exploring. Early physical transformation methods often resulted in high-resolution images with significant information loss, and the edges and details were not well recovered.With advancements in hardware technology and mathematics, deep learning methods have been employed for image super-resolution tasks. These range from direct deep learning models, residual channel attention networks, bi-directional suppression networks, to networks with transformer modules, all of which have progressively yielded satisfactory results.In the realm of multi-image super-resolution, the establishment of a multi-image super-resolution dataset has facilitated the evolution from convolution models to transformer models, thereby continuously enhancing the quality of super-resolution. However, it has been observed that neither pure convolution nor pure transformer networks can effectively utilize low-resolution image information.To address this, we propose a novel end-to-end CoT-MISR network. The CoT-MISR network compensates for local and global information by leveraging the strengths of both convolution and transformer techniques. Validation on an equal parameter dataset demonstrates that our CoT-MISR network has achieved the optimal score index.