Modality specific infrared and visible image fusion based on multi-scale rich feature representation under low-light environment

• Published in: Infrared physics & technology Vol. 140; p. 105351
• Main Authors: Liu, Chenhua, Chen, Hanrui, Deng, Lei, Guo, Chentong, Lu, Xitian, Yu, Heng, Zhu, Lianqing, Dong, Mingli
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.08.2024
• Subjects: Human visual perception; Image fusion; Infrared images; Low-light image enhancement; Superpixel; Superpixel; Infrared images; Image fusion; Human visual perception; Low-light image enhancement
• ISSN: 1350-4495; 1879-0275
• DOI: 10.1016/j.infrared.2024.105351

Infrared (IR) and Visible (VIS) image fusion is a critical task in computer vision, which aims to produce high-quality fused images by combining two modal images with complementary information. However, current image fusion techniques neglect the severe degradation of texture and details in low-light environments and do not account for human visual perception. In this paper, a multi-scale image fusion algorithm adapted to the low-light environment is proposed. Firstly, the light distribution of the low-light image is estimated and adjusted based on the gray value probability distribution, resulting in an enhanced VIS image with rich details. Simultaneously, considering the imaging characteristics of the two modal images, a superpixel method is utilized to obtain a saliency map for the IR image, and a multi-scale feature extraction scheme is employed to establish a VIS detail map. Finally, we employ entropy to measure the amount of information in the saliency and detail maps to guide the fusion process. Experimental results from qualitative and quantitative evaluations demonstrate that the fused image conforms to human visual perception, maintains a uniform distribution of pixel intensity, and preserves rich texture features at the edge details.