Co-visual pattern augmented generative transformer learning for automobile geo-localization

TL;DR

This paper introduces a novel transformer-based method with co-visual pattern augmentation for cross-view geo-localization, significantly improving accuracy in matching aerial and ground images for vehicle localization.

Contribution

It proposes a mutual generative transformer learning framework that leverages cross-view knowledge generation and cascaded attention masking to enhance geo-localization accuracy.

Findings

Sets new state-of-the-art results on CVACT and CVUSA benchmarks.

Demonstrates the effectiveness of mutual generative transformers in cross-view matching.

Improves accuracy over existing Siamese-like architectures.

Abstract

Geolocation is a fundamental component of route planning and navigation for unmanned vehicles, but GNSS-based geolocation fails under denial-of-service conditions. Cross-view geo-localization (CVGL), which aims to estimate the geographical location of the ground-level camera by matching against enormous geo-tagged aerial (\emph{e.g.}, satellite) images, has received lots of attention but remains extremely challenging due to the drastic appearance differences across aerial-ground views. In existing methods, global representations of different views are extracted primarily using Siamese-like architectures, but their interactive benefits are seldom taken into account. In this paper, we present a novel approach using cross-view knowledge generative techniques in combination with transformers, namely mutual generative transformer learning (MGTL), for CVGL. Specifically, by taking the initial representations produced by the backbone network, MGTL develops two separate generative sub-modules -- one for aerial-aware knowledge generation from ground-view semantics and vice versa -- and fully exploits the entirely mutual benefits through the attention mechanism. Moreover, to better capture the co-visual relationships between aerial and ground views, we introduce a cascaded attention masking algorithm to further boost accuracy. Extensive experiments on challenging public benchmarks, \emph{i.e.}, {CVACT} and {CVUSA}, demonstrate the effectiveness of the proposed method which sets new records compared with the existing state-of-the-art models.