Render-and-Compare: Cross-View 6 DoF Localization from Noisy Prior

TL;DR

This paper introduces a novel cross-view localization method from aerial to ground images using a render-and-compare pipeline, addressing the lack of existing datasets and outperforming state-of-the-art baselines.

Contribution

It proposes a new cross-view 6-DoF localization approach with an iterative render-and-compare method and a semi-automatic dataset collection system.

Findings

Our method outperforms existing approaches significantly.

We created a new dataset for cross-view localization.

The approach enhances robustness with noisy initial priors.

Abstract

Despite the significant progress in 6-DoF visual localization, researchers are mostly driven by ground-level benchmarks. Compared with aerial oblique photography, ground-level map collection lacks scalability and complete coverage. In this work, we propose to go beyond the traditional ground-level setting and exploit the cross-view localization from aerial to ground. We solve this problem by formulating camera pose estimation as an iterative render-and-compare pipeline and enhancing the robustness through augmenting seeds from noisy initial priors. As no public dataset exists for the studied problem, we collect a new dataset that provides a variety of cross-view images from smartphones and drones and develop a semi-automatic system to acquire ground-truth poses for query images. We benchmark our method as well as several state-of-the-art baselines and demonstrate that our method outperforms other approaches by a large margin.