C-BEV: Contrastive Bird's Eye View Training for Cross-View Image Retrieval and 3-DoF Pose Estimation

TL;DR

This paper introduces C-BEV, a contrastive learning approach using bird's eye view maps for cross-view image retrieval and 3-DoF pose estimation, significantly improving accuracy in challenging real-world scenarios.

Contribution

It proposes a novel BEV-based retrieval architecture that addresses many-to-one ambiguities and learns 3-DoF pose estimation without explicit metric supervision.

Findings

Surpasses state-of-the-art in cross-view retrieval accuracy.

Doubles top-1 recall in challenging scenarios.

Learns accurate 3-DoF pose estimation without direct groundtruth.

Abstract

To find the geolocation of a street-view image, cross-view geolocalization (CVGL) methods typically perform image retrieval on a database of georeferenced aerial images and determine the location from the visually most similar match. Recent approaches focus mainly on settings where street-view and aerial images are preselected to align w.r.t. translation or orientation, but struggle in challenging real-world scenarios where varying camera poses have to be matched to the same aerial image. We propose a novel trainable retrieval architecture that uses bird's eye view (BEV) maps rather than vectors as embedding representation, and explicitly addresses the many-to-one ambiguity that arises in real-world scenarios. The BEV-based retrieval is trained using the same contrastive setting and loss as classical retrieval. Our method C-BEV surpasses the state-of-the-art on the retrieval task on multiple datasets by a large margin. It is particularly effective in challenging many-to-one scenarios, e.g. increasing the top-1 recall on VIGOR's cross-area split with unknown orientation from 31.1% to 65.0%. Although the model is supervised only through a contrastive objective applied on image pairings, it additionally learns to infer the 3-DoF camera pose on the matching aerial image, and even yields a lower mean pose error than recent methods that are explicitly trained with metric groundtruth.