Boosting 3-DoF Ground-to-Satellite Camera Localization Accuracy via Geometry-Guided Cross-View Transformer

TL;DR

This paper introduces a geometry-guided cross-view transformer that significantly enhances ground-to-satellite camera localization accuracy by effectively estimating relative rotation and translation, outperforming existing methods on benchmark datasets.

Contribution

It proposes a novel geometry-guided cross-view transformer and neural pose optimizer for precise localization, integrating geometric and learnable features for improved accuracy.

Findings

Likelihood of lateral pose within 1m increased from 35.54% to 76.44%.

Orientation within 1° improved from 19.64% to 99.10%.

Method outperforms state-of-the-art on KITTI dataset.

Abstract

Image retrieval-based cross-view localization methods often lead to very coarse camera pose estimation, due to the limited sampling density of the database satellite images. In this paper, we propose a method to increase the accuracy of a ground camera's location and orientation by estimating the relative rotation and translation between the ground-level image and its matched/retrieved satellite image. Our approach designs a geometry-guided cross-view transformer that combines the benefits of conventional geometry and learnable cross-view transformers to map the ground-view observations to an overhead view. Given the synthesized overhead view and observed satellite feature maps, we construct a neural pose optimizer with strong global information embedding ability to estimate the relative rotation between them. After aligning their rotations, we develop an uncertainty-guided spatial correlation to generate a probability map of the vehicle locations, from which the relative translation can be determined. Experimental results demonstrate that our method significantly outperforms the state-of-the-art. Notably, the likelihood of restricting the vehicle lateral pose to be within 1m of its Ground Truth (GT) value on the cross-view KITTI dataset has been improved from $35.54\%$ to $76.44\%$, and the likelihood of restricting the vehicle orientation to be within $1^{\circ}$ of its GT value has been improved from $19.64\%$ to $99.10\%$.