Multiple-environment Self-adaptive Network for aerial-view geo-localization

• Published in: Pattern recognition Vol. 152; p. 110363
• Main Authors: Wang, Tingyu, Zheng, Zhedong, Sun, Yaoqi, Yan, Chenggang, Yang, Yi, Chua, Tat-Seng
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.08.2024
• Subjects: Cross-view geo-localization; Deep learning; Image retrieval; Multi-platform collaboration; Multi-source domain generalization; Deep learning; Multi-platform collaboration; Multi-source domain generalization; Cross-view geo-localization; Image retrieval
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2024.110363

Aerial-view geo-localization tends to determine an unknown position through matching the drone-view image with the geo-tagged satellite-view image. This task is mostly regarded as an image retrieval problem. The key underpinning this task is to design a series of deep neural networks to learn discriminative image descriptors. However, existing methods meet large performance drops under realistic weather, such as rain and fog, since they do not take the domain shift between the training data and multiple test environments into consideration. To minor this domain gap, we propose a Multiple-environment Self-adaptive Network (MuSe-Net) to dynamically adjust the domain shift caused by environmental changing. In particular, MuSe-Net employs a two-branch neural network containing one multiple-environment style extraction network and one self-adaptive feature extraction network. As the name implies, the multiple-environment style extraction network is to extract the environment-related style information, while the self-adaptive feature extraction network utilizes an adaptive modulation module to dynamically minimize the environment-related style gap. Extensive experiments on three widely-used benchmarks, i.e., University-1652, SUES-200, and CVUSA, demonstrate that the proposed MuSe-Net achieves a competitive result for geo-localization in multiple environments. Furthermore, we observe that the proposed method also shows great potential to the unseen extreme weather, such as mixing the fog, rain and snow. •Identifying one key challenge in visual geo-localization: weather and illumination changes.•Presenting MuSe-Net to alleviate the interference caused by environmental changes.•Designing Residual SPADE for efficient training and feature discrimination boosting.•Results on three geo-localization benchmarks confirm the superiority of our MuSe-Net.