Infrared and Visible Image Fusion via Texture Conditional Generative Adversarial Network

• Published in: IEEE transactions on circuits and systems for video technology Vol. 31; no. 12; pp. 4771 - 4783
• Main Authors: Yang, Yong, Liu, Jiaxiang, Huang, Shuying, Wan, Weiguo, Wen, Wenying, Guan, Juwei
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Codec; combined texture map; Computer vision; Feature extraction; Generative adversarial networks; Generators; Image fusion; Image processing; Information filters; Infrared and visible image fusion; Infrared imagery; Infrared imaging; multiple decision maps; Night vision; TC-GAN; Texture mapping; Training data
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2021.3054584

This paper proposes an effective infrared and visible image fusion method based on a texture conditional generative adversarial network (TC-GAN). The constructed TC-GAN generates a combined texture map for capturing gradient changes in image fusion. The generator in the TC-GAN is designed as a codec structure for extracting more details, and a squeeze-and-excitation module is applied to this codec structure to increase the weight of significant texture information in the combined texture map. The generator loss function is designed by combing the gradient loss and adversarial loss to retain the texture information of the source images. The discriminator brings the texture of the generated image closer to the visible image. To obtain significant texture information from the source images, a multiple decision map-based fusion strategy is proposed using a combined texture map and an adaptive guided filter. Extensive experiments on the public TNO and RoadScene datasets demonstrate that the proposed method is superior to other state-of-the-art algorithms in terms of a subjective evaluation and quantitative indicators.