GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization

TL;DR

GSplatLoc introduces a novel 3D Gaussian Splatting-based approach that integrates keypoint descriptors for improved visual localization accuracy in diverse environments, outperforming recent neural rendering methods.

Contribution

The paper presents a new two-stage localization method combining 3D Gaussian Splatting with keypoint descriptors for enhanced accuracy and robustness.

Findings

Outperforms recent neural rendering localization methods

Effective in both indoor and outdoor environments

Improves pose estimation accuracy

Abstract

Although various visual localization approaches exist, such as scene coordinate regression and camera pose regression, these methods often struggle with optimization complexity or limited accuracy. To address these challenges, we explore the use of novel view synthesis techniques, particularly 3D Gaussian Splatting (3DGS), which enables the compact encoding of both 3D geometry and scene appearance. We propose a two-stage procedure that integrates dense and robust keypoint descriptors from the lightweight XFeat feature extractor into 3DGS, enhancing performance in both indoor and outdoor environments. The coarse pose estimates are directly obtained via 2D-3D correspondences between the 3DGS representation and query image descriptors. In the second stage, the initial pose estimate is refined by minimizing the rendering-based photometric warp loss. Benchmarking on widely used indoor and outdoor datasets demonstrates improvements over recent neural rendering-based localization methods, such as NeRFMatch and PNeRFLoc.