Learning to Localize in New Environments from Synthetic Training Data

TL;DR

This paper introduces a novel learning-based visual localization method that generalizes to new environments using synthetic training data, hierarchical correlation, and uncertainty modeling, outperforming classical and existing learning approaches.

Contribution

The authors propose a scene-agnostic localization approach that leverages synthetic data and architectural modifications to improve generalization and accuracy in unknown environments.

Findings

Outperforms 5-point algorithm with SIFT features on large images.

Surpasses previous learning-based localization methods trained on different data.

Excels in scenarios with limited reference images, outperforming classical methods.

Abstract

Most existing approaches for visual localization either need a detailed 3D model of the environment or, in the case of learning-based methods, must be retrained for each new scene. This can either be very expensive or simply impossible for large, unknown environments, for example in search-and-rescue scenarios. Although there are learning-based approaches that operate scene-agnostically, the generalization capability of these methods is still outperformed by classical approaches. In this paper, we present an approach that can generalize to new scenes by applying specific changes to the model architecture, including an extended regression part, the use of hierarchical correlation layers, and the exploitation of scale and uncertainty information. Our approach outperforms the 5-point algorithm using SIFT features on equally big images and additionally surpasses all previous learning-based approaches that were trained on different data. It is also superior to most of the approaches that were specifically trained on the respective scenes. We also evaluate our approach in a scenario where only very few reference images are available, showing that under such more realistic conditions our learning-based approach considerably exceeds both existing learning-based and classical methods.