FaVoR: Features via Voxel Rendering for Camera Relocalization

TL;DR

FaVoR introduces a voxel rendering-based feature representation for camera relocalization, improving robustness to view changes and outperforming existing methods in indoor environments with lower resource demands.

Contribution

The paper presents a novel voxel rendering approach that generates view-invariant features for camera relocalization, addressing limitations of traditional feature matching under viewpoint changes.

Findings

Up to 39% improvement in median translation error on indoor datasets

Outperforms state-of-the-art feature methods in indoor environments

Maintains lower memory and computational costs for outdoor scenarios

Abstract

Camera relocalization methods range from dense image alignment to direct camera pose regression from a query image. Among these, sparse feature matching stands out as an efficient, versatile, and generally lightweight approach with numerous applications. However, feature-based methods often struggle with significant viewpoint and appearance changes, leading to matching failures and inaccurate pose estimates. To overcome this limitation, we propose a novel approach that leverages a globally sparse yet locally dense 3D representation of 2D features. By tracking and triangulating landmarks over a sequence of frames, we construct a sparse voxel map optimized to render image patch descriptors observed during tracking. Given an initial pose estimate, we first synthesize descriptors from the voxels using volumetric rendering and then perform feature matching to estimate the camera pose. This methodology enables the generation of descriptors for unseen views, enhancing robustness to view changes. We extensively evaluate our method on the 7-Scenes and Cambridge Landmarks datasets. Our results show that our method significantly outperforms existing state-of-the-art feature representation techniques in indoor environments, achieving up to a 39% improvement in median translation error. Additionally, our approach yields comparable results to other methods for outdoor scenarios while maintaining lower memory and computational costs.