MOFA: A novel dataset for Multi-modal Image Fusion Applications

• Published in: Information fusion Vol. 96; pp. 144 - 155
• Main Authors: Xiao, Kaihua, Kang, Xudong, Liu, Haibo, Duan, Puhong
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.08.2023
• Subjects: Image fusion; Long-wavelength infrared (LWIR); Multi-modal dataset; Near-infrared (NIR); Polarization; Polarization; Long-wavelength infrared (LWIR); Multi-modal dataset; Near-infrared (NIR); Image fusion
• ISSN: 1566-2535; 1872-6305
• DOI: 10.1016/j.inffus.2023.03.012

Multi-modal image fusion is widely used in various fields, especially in military, medical, industrial detection and other fields. Image fusion can integrate redundant and complementary information of two or more multi-modal images into one image, so that the fused image contains more useful information. In this paper, we construct a novel dataset for Multi-modal Image Fusion Applications (MOFA), including four modals: visible, near-infrared (NIR), long-wavelength infrared (LWIR) and polarization. The MOFA dataset contains 1062 images of 118 groups, in which 450 are indoor and 612 are outdoor. The dataset is applied to different image fusion applications, including general multi-modal image fusion, fusion based image super-resolution and image restoration. Multiple image fusion methods are compared and analyzed on this dataset with a qualitative assessment of subjective and objective metrics. Based on the experiments, the advantages and disadvantages of different methods are discussed. Moreover, the challenging problems of image fusion are concluded. •Build a multi-modal imaging device.•Construct a multi-modal dataset, including visible, NIR, LWIR and polarization.•Explore the effectiveness of the dataset for different image processing tasks.•Analyze the advantages and disadvantages of different fusion methods.