HBANet: A hybrid boundary-aware attention network for infrared and visible image fusion

• Published in: Computer vision and image understanding Vol. 249; p. 104161
• Main Authors: Luo, Xubo, Zhang, Jinshuo, Wang, Liping, Niu, Dongmei
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.12.2024
• Subjects: Boundary prior; Cross attention; Deep learning; Image Fusion; Deep learning; Boundary prior; Cross attention; Image Fusion; 41A10; 65D05; 65D17; 41A05
• ISSN: 1077-3142
• DOI: 10.1016/j.cviu.2024.104161

Infrared and visible image fusion is an extensively investigated problem in infrared image processing, aiming to extract useful information from source images. However, the automatic fusion of these images presents a significant challenge due to the large domain difference and ambiguous boundaries. In this article, we propose a novel image fusion approach based on hybrid boundary-aware attention, termed HBANet, which models global dependencies across the image and leverages boundary-wise prior knowledge to supplement local details. Specifically, we design a novel mixed boundary-aware attention module that is capable of leveraging spatial information to the fullest extent and integrating long dependencies across different domains. To preserve the integrity of texture and structural information, we introduced a sophisticated loss function that comprises structure, intensity, and variation losses. Our method has been demonstrated to outperform state-of-the-art methods in terms of both visual and quantitative metrics, in our experiments on public datasets. Furthermore, our approach also exhibits great generalization capability, achieving satisfactory results in CT and MRI image fusion tasks. •A novel infrared and visible image fusion framework, HBANet, is proposed to fuse images while paying more attention to the edge areas.•A boundary-aware attention module is proposed to drive the model to fully utilize the boundary-wise prior.•A cross-domain attention module is designed to integrate long dependencies across different domains.•To preserve the featured information of source images, we proposed a hybrid loss function to constrain the HBANet.