S2H-GNN: Learning Soft to Hard Feature Matching with Sparsified Graph Neural Network

• Published in: 2023 IEEE International Conference on Real-time Computing and Robotics (RCAR) pp. 756 - 761
• Main Authors: Xie, Tao, Li, Ruifeng, Jiang, Zhiqiang, Dai, Kun, Wang, Ke, Zhao, Lijun, Li, Shuozhan
• Format: Conference Proceeding
• Language: English
• Published: IEEE 17.07.2023
• Subjects: Estimation; Graph neural networks; Image matching; Message passing; Multiplexing; Redundancy; Transformers
• DOI: 10.1109/RCAR58764.2023.10249019

In this work, we introduce a soft-to-hard network (S2H-GNN) for precise local feature matching between image pairs. The core of S2H-GNN is a weighted context aggregation algorithm that enables nodes to receive information from their neighbors in order to form a more global understanding. S2HGNN leverages a sparsified graph neural network to enhance the features, consisting of the following modules alternately: (1) Inducing Point Initial Module that uses cosine correlation to identify highly scored keypoints as Inducing points. (2) Message Passing Module that integrates the features of original keypoints into those of Inducing points, which are further leveraged to exchange messages through self attention and cross attention. (3) Attention Upsample Module that propagates the feature of Inducing points to that of original keypoints, obtaining discriminative feature representation. Additionally, we propose an assignment layer to obtain a soft assignment matrix, which views feature matching as a graphy matching problem. Finally, we introduce a novel gap loss to guide S2H-GNN in learning a discriminative cognition of matching and non-matching keypoints, ultimately obtaining the hard assignment matrix. Experimental results demonstrate that S2H-GNN achieves comparable or higher performance in image matching and homography estimation tasks.