GLS–MIFT: A modality invariant feature transform with global-to-local searching

• Published in: Information fusion Vol. 105; p. 102252
• Main Authors: Fan, Zhongli, Pi, Yingdong, Wang, Mi, Kang, Yifei, Tan, Kai
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.05.2024
• Subjects: Descriptor and matcher; Feature detector; Global-to-local searching (GLS) strategy; Multimodal images; Nonlinear radiation distortion (NRD); Scale- and rotation-invariance; Multimodal images; Scale- and rotation-invariance; Global-to-local searching (GLS) strategy; Nonlinear radiation distortion (NRD); Descriptor and matcher; Feature detector
• DOI: 10.1016/j.inffus.2024.102252
• ISSN: 1566-2535

•Generic feature transform for multimodal images in a modality and geometric invariant manner.•High repeatability feature detection by introducing an effective feature response aggregation model.•Robust rotation invariant feature description based on a well-designed feature descriptor and a novel primary direction estimation approach.•Achieving highly reliable feature matching by designing a novel feature matcher based on the global-to-local searching principle. Accurate image matching is the basis for many information fusion-related applications. Conventional methods fail when handling multimodal image pairs with severe nonlinear radiation distortion (NRD) and geometric transformations. To address this problem, we present an effective method, termed modality-invariant feature transform with global-to-local searching (GLS–MIFT). First, we addressed scale changes by constructing the image scale space. Then, we obtained multi-scale, multi-orientation filtering results based on first-order Gaussian steerable filters to exploit their robustness to NRD. Next, we introduced a feature response aggregation model to synthesize the filtering results to generate a feature map, and used it to detect highly repeatable features. Additionally, we designed an adaptive partitioning descriptor to achieve rotation-invariant feature description, involving the following six steps: generation of statistical histograms of multi-orientation filtering values, synthesis of histograms on multiple scales, estimation and updating of the primary direction, determination of the sampling direction, normalization of the feature vector in sub-regions, and finally, obtaining the complete description vector. A convolutional image grouping strategy was used to enhance the rotational invariance of the method. We developed a new feature matcher based on the GLS strategy. Guided by the results of global searching stage, the local searching stage further improved the matching accuracy and reliability of the results. Our experimental results confirmed that GLS–MIFT achieved high-quality matching for a large-scale dataset of 1110 image pairs, with various multimodal image types from the fields of computer vision, medicine, and remote sensing. GLS–MIFT outperformed state-of-the-art methods including SIFT, RIFT, HOWP, OFM, MatchFormer, SemLA and GIFT in qualitative and quantitative evaluations. Our implementation and datasets are available at: https://github.com/Zhongli-Fan/GLS-MIFT.