Cross-UNet: dual-branch infrared and visible image fusion framework based on cross-convolution and attention mechanism

• Published in: The Visual computer Vol. 39; no. 10; pp. 4801 - 4818
• Main Authors: Wang, Xuejiao, Hua, Zhen, Li, Jinjiang
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.10.2023; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Computer Graphics; Computer Science; Computer vision; Convolution; Decomposition; Deep learning; Dictionaries; Image Processing and Computer Vision; Infrared imagery; Methods; Neural networks; Original Article; Sensors; Attention mechanism; Cross-convolution; Image fusion
• ISSN: 0178-2789; 1432-2315
• DOI: 10.1007/s00371-022-02628-6

Existing infrared and visible image fusion methods suffer from edge information loss, artifact introduction, and image distortion. Therefore, a dual-branch network model based on the attention mechanism, Cross-UNet, is proposed in this paper for infrared and visible image fusion. First, the encoder part adopts an asymmetric convolution kernel, which can simultaneously obtain local detail information and global structural information of the source image from different directions. Second, in order to fuse the dual-branch image features of different scales, a dual-attention mechanism is added to the fusion block. Finally, the decoder adopts an attention model with a large receptive field to enhance the ability to judge the importance of features, thereby improving the fusion quality. On the public datasets of TNO, RoadScene, and Country, the results are fully compared with nine other advanced fusion methods both qualitatively and quantitatively. The results show that the model in this paper has superior performance and high stability.