RTHEN: Unsupervised deep homography estimation based on dynamic attention for repetitive texture image stitching

• Published in: Displays Vol. 82; p. 102670
• Main Authors: Yan, Ni, Mei, Yupeng, Yang, Tian, Yu, Huihui, Chen, Yingyi
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.04.2024
• Subjects: Deep learning; Dynamic attention; Homography estimation; Repetitive textures; Triplet loss; Deep learning; Repetitive textures; Dynamic attention; Triplet loss; Homography estimation
• ISSN: 0141-9382; 1872-7387
• DOI: 10.1016/j.displa.2024.102670

Homography estimation is regarded as one of the key challenges in image alignment, where the goal is to estimate the projective transformation between two images on the same plane. Unsupervised learning methods are gradually becoming popular due to their excellent performance and lack of need for labeled data. However, in regional scenes with repeated textures, there may be ambiguity in the correspondence between local features, affecting homography estimation accuracy. This paper proposes a new unsupervised deep homography method RTHEN to solve such problems. In order to effectively obtain repeated texture features, a multi-scale Feature pyramid Siamese network (FPSN) is designed. Specifically, we dynamically allocate the weights of recited texture features through a dynamic attention module and introduce a channel attention module to provide rich contextual information for repeated texture areas. We propose a hard triplet loss function based on overlap constraints to optimize the matching results. At the same time, we collected a repetitive texture image dataset (RTID) for homography estimation training and evaluation. Experimental results show that our method outperforms existing learning methods in repetitive texture scenes and offers competitive performance with state-of-the-art traditional methods. •An unsupervised method is employed for homography estimation.•The issue of mismatching in repetitive textures is effectively addressed.•Formulated a loss function with overlap constraints to enhance estimation results.•Curated a dataset of repeated texture images for training and model evaluation.