TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization

• Published in: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1152 - 1161
• Main Authors: Zhu, Sijie, Shah, Mubarak, Chen, Chen
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2022
• Subjects: categorization; Computer vision; Correlation; Costs; Image recognition; Image resolution; Recognition: detection; retrieval; Photogrammetry and remote sensing; Transformers; Transforms
• ISSN: 2575-7075
• DOI: 10.1109/CVPR52688.2022.00123

The dominant CNN-based methods for cross-view image geo-localization rely on polar transform and fail to model global correlation. We propose a pure transformer-based approach (TransGeo) to address these limitations from a different perspective. TransGeo takes full advantage of the strengths of transformer related to global information modeling and explicit position information encoding. We further leverage the flexibility of transformer input and propose an attention-guided non-uniform cropping method, so that uninformative image patches are removed with negligible drop on performance to reduce computation cost. The saved computation can be reallocated to increase resolution only for informative patches, resulting in performance improvement with no additional computation cost. This "attend and zoom-in" strategy is highly similar to human behavior when observing images. Remarkably, TransGeo achieves state-of-the-art results on both urban and rural datasets, with significantly less computation cost than CNN-based methods. It does not rely on polar transform and infers faster than CNN-based methods. Code is available at https://github.com/Jeff-Zilence/TransGeo2022.