Transformer-Guided Convolutional Neural Network for Cross-View Geolocalization

TL;DR

This paper introduces TransGCNN, a novel Transformer-guided CNN architecture that improves cross-view geolocalization accuracy by combining local CNN features with global Transformer representations, achieving state-of-the-art results efficiently.

Contribution

The paper proposes a new Transformer-guided CNN model that enhances feature discrimination for ground-to-aerial geolocalization, with a dual-branch Transformer head for multi-scale global feature integration.

Findings

Achieves top-1 accuracy of 94.12% on CVUSA dataset.

Outperforms baselines with fewer parameters and higher frame rate.

Demonstrates superior accuracy-efficiency tradeoff.

Abstract

Ground-to-aerial geolocalization refers to localizing a ground-level query image by matching it to a reference database of geo-tagged aerial imagery. This is very challenging due to the huge perspective differences in visual appearances and geometric configurations between these two views. In this work, we propose a novel Transformer-guided convolutional neural network (TransGCNN) architecture, which couples CNN-based local features with Transformer-based global representations for enhanced representation learning. Specifically, our TransGCNN consists of a CNN backbone extracting feature map from an input image and a Transformer head modeling global context from the CNN map. In particular, our Transformer head acts as a spatial-aware importance generator to select salient CNN features as the final feature representation. Such a coupling procedure allows us to leverage a lightweight Transformer network to greatly enhance the discriminative capability of the embedded features. Furthermore, we design a dual-branch Transformer head network to combine image features from multi-scale windows in order to improve details of the global feature representation. Extensive experiments on popular benchmark datasets demonstrate that our model achieves top-1 accuracy of 94.12\% and 84.92\% on CVUSA and CVACT_val, respectively, which outperforms the second-performing baseline with less than 50% parameters and almost 2x higher frame rate, therefore achieving a preferable accuracy-efficiency tradeoff.