Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution

• Published in: Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 20604 - 20613
• Main Authors: Li, Guangyuan, Lv, Jun, Tian, Yapeng, Dou, Qi, Wang, Chengyan, Xu, Chenliang, Qin, Jing
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.01.2022
• Subjects: Aggregates; Anatomical structure; biological and cell microscopy; Image and video synthesis and generation; Codes; Computer vision; Fuses; Magnetic resonance imaging; Medical; Superresolution
• ISSN: 1063-6919
• DOI: 10.1109/CVPR52688.2022.01998

Magnetic resonance imaging (MRI) can present multicontrast images of the same anatomical structures, enabling multi-contrast super-resolution (SR) techniques. Compared with SR reconstruction using a single-contrast, multicontrast SR reconstruction is promising to yield SR images with higher quality by leveraging diverse yet complementary information embedded in different imaging modalities. However, existing methods still have two shortcomings: (1) they neglect that the multi-contrast features at different scales contain different anatomical details and hence lack effective mechanisms to match and fuse these features for better reconstruction; and (2) they are still deficient in capturing long-range dependencies, which are essential for the regions with complicated anatomical structures. We propose a novel network to comprehensively address these problems by developing a set of innovative Transformer-empowered multi-scale contextual matching and aggregation techniques; we call it McMRSR. Firstly, we tame transformers to model long-range dependencies in both reference and target images. Then, a new multi-scale contextual matching method is proposed to capture corresponding contexts from reference features at different scales. Furthermore, we introduce a multi-scale aggregation mechanism to gradually and interactively aggregate multi-scale matched features for reconstructing the target SR MR image. Extensive experiments demonstrate that our network outperforms state-of-the-art approaches and has great potential to be applied in clinical practice. Codes are available at https://github.com/XAIMI-Lab/McMRSR.