Accelerated Coordinate Encoding: Learning to Relocalize in Minutes using RGB and Poses

TL;DR

This paper introduces a novel relocalization method that achieves high accuracy in under 5 minutes by using a scene-agnostic backbone and a scene-specific MLP head, enabling rapid training without additional scene data.

Contribution

It presents a fast training approach for visual relocalization using a scene-agnostic backbone and a scene-specific MLP head, reducing training time from hours to minutes.

Findings

Achieves comparable accuracy to state-of-the-art methods.

Reduces training time by up to 300x.

Does not require depth maps or 3D models for training.

Abstract

Learning-based visual relocalizers exhibit leading pose accuracy, but require hours or days of training. Since training needs to happen on each new scene again, long training times make learning-based relocalization impractical for most applications, despite its promise of high accuracy. In this paper we show how such a system can actually achieve the same accuracy in less than 5 minutes. We start from the obvious: a relocalization network can be split in a scene-agnostic feature backbone, and a scene-specific prediction head. Less obvious: using an MLP prediction head allows us to optimize across thousands of view points simultaneously in each single training iteration. This leads to stable and extremely fast convergence. Furthermore, we substitute effective but slow end-to-end training using a robust pose solver with a curriculum over a reprojection loss. Our approach does not require privileged knowledge, such a depth maps or a 3D model, for speedy training. Overall, our approach is up to 300x faster in mapping than state-of-the-art scene coordinate regression, while keeping accuracy on par.