FocusTune: Tuning Visual Localization through Focus-Guided Sampling

TL;DR

FocusTune introduces a focus-guided sampling method that enhances visual localization accuracy by concentrating training on geometrically critical regions, achieving state-of-the-art results with low computational costs.

Contribution

The paper presents a novel focus-guided sampling strategy for scene coordinate regression, improving localization performance while maintaining low resource requirements.

Findings

Reduces translation error from 25cm to 19cm on Cambridge Landmarks.

Improves localization accuracy with minimal additional computational cost.

Compatible with ACE model, enhancing its performance without increasing complexity.

Abstract

We propose FocusTune, a focus-guided sampling technique to improve the performance of visual localization algorithms. FocusTune directs a scene coordinate regression model towards regions critical for 3D point triangulation by exploiting key geometric constraints. Specifically, rather than uniformly sampling points across the image for training the scene coordinate regression model, we instead re-project 3D scene coordinates onto the 2D image plane and sample within a local neighborhood of the re-projected points. While our proposed sampling strategy is generally applicable, we showcase FocusTune by integrating it with the recently introduced Accelerated Coordinate Encoding (ACE) model. Our results demonstrate that FocusTune both improves or matches state-of-the-art performance whilst keeping ACE's appealing low storage and compute requirements, for example reducing translation error from 25 to 19 and 17 to 15 cm for single and ensemble models, respectively, on the Cambridge Landmarks dataset. This combination of high performance and low compute and storage requirements is particularly promising for applications in areas like mobile robotics and augmented reality. We made our code available at \url{https://github.com/sontung/focus-tune}.