CT and MRI image fusion via multimodal feature interaction network

• Published in: Network modeling and analysis in health informatics and bioinformatics (Wien) Vol. 13; no. 1; p. 13
• Main Authors: Song, Wenhao, Zeng, Xiangqin, Li, Qilei, Gao, Mingliang, Zhou, Hui, Shi, Junzhi
• Format: Journal Article
• Language: English
• Published: Vienna Springer Vienna 27.03.2024; Springer Nature B.V
• Subjects: Adaptation; Applications of Graph Theory and Complex Networks; Bioinformatics; Computational Biology/Bioinformatics; Computed tomography; Computer Science; Computer vision; Decomposition; Deep learning; Design; Feature extraction; Health Informatics; Image enhancement; Image processing; Image quality; Image reconstruction; Information processing; Magnetic resonance imaging; Measurement techniques; Medical diagnosis; Medical imaging; Methods; Original Article; Wavelet transforms; Gated mechanism; Healthcare; CT/MRI image; Image fusion
• ISSN: 2192-6670; 2192-6662; 2192-6670
• DOI: 10.1007/s13721-024-00449-2

Computed tomography (CT) and magnetic resonance imaging (MRI) image fusion is a popular technique for integrating information from two different modalities of medical images. This technique can improve image quality and diagnostic efficacy. To effectively extract and balance complementary information in the source images, we propose an end-to-end multimodal feature interaction network (MFINet) to fuse CT and MRI images. The MIFNet consists of a shallow feature extractor, a feature interaction (FI), and an image reconstruction. In the FI, we design a deep feature extraction module, which consists of a series of gated feature enhancement units (GFEUs) and convolutional layers. To extract key features from images, we introduce a gated normalization block in the GFEU, which can achieve feature selection. Comprehensive experiments demonstrate that the proposed end-to-end fusion network outperforms existing state-of-the-art methods in both qualitative and quantitative assessments.