GSLoc: Visual Localization with 3D Gaussian Splatting

TL;DR

GSLoc introduces a dense visual localization method using 3D Gaussian Splatting that improves accuracy in challenging, textureless environments by leveraging a novel map representation and a coarse-to-fine alignment strategy.

Contribution

The paper proposes GSLoc, a novel localization approach that employs 3D Gaussian Splatting and backpropagation of pose gradients, outperforming neural sparse methods in difficult scenarios.

Findings

Effective in textureless environments with small overlap

Outperforms neural sparse methods in challenging conditions

Enhances localization by mixing real and virtual keyframes

Abstract

We present GSLoc: a new visual localization method that performs dense camera alignment using 3D Gaussian Splatting as a map representation of the scene. GSLoc backpropagates pose gradients over the rendering pipeline to align the rendered and target images, while it adopts a coarse-to-fine strategy by utilizing blurring kernels to mitigate the non-convexity of the problem and improve the convergence. The results show that our approach succeeds at visual localization in challenging conditions of relatively small overlap between initial and target frames inside textureless environments when state-of-the-art neural sparse methods provide inferior results. Using the byproduct of realistic rendering from the 3DGS map representation, we show how to enhance localization results by mixing a set of observed and virtual reference keyframes when solving the image retrieval problem. We evaluate our method both on synthetic and real-world data, discussing its advantages and application potential.