Disambiguating 2D-3D Correspondences in Gaussian Splatting-based Feature Fields for Visual Localization

TL;DR

This paper introduces SplitGS-Loc, a novel framework that disambiguates 2D-3D correspondences in Gaussian Splatting-based Feature Fields to improve visual localization accuracy and stability.

Contribution

It proposes a Gaussian splitting method and feature aggregation strategy that enhance the multi-view consistency and discriminative power of GSFFs for localization.

Findings

Achieves state-of-the-art localization performance on benchmarks.

Enables stable PnP convergence with photometric GSFFs.

Does not require per-scene training or iterative pose refinement.

Abstract

While Gaussian Splatting-based Feature Fields (GSFFs) have shown promise for visual localization, this paper highlights that photometrically optimized GSFFs are inherently ill-suited for 2D-3D matching. The volumetric extent of each Gaussian induces many-to-one pixel-to-point mappings that destabilize PnP-based pose estimation, while photometric optimization gives rise to superfluous Gaussians devoid of multi-view consistency. To address these issues, we propose SplitGS-Loc, a localization-specialized GSFFs construction framework that disambiguates 2D-3D correspondences by exploiting Gaussian attributes. Our key design, Mixture-of-Gaussians-based splitting, decomposes each Gaussian into smaller Gaussians, replacing ambiguous many-to-one with precise one-to-one correspondences. In parallel, we exploit composition weights from GS rasterization to select Gaussians that significantly and consistently contribute across multiple views and aggregate discriminative features through strong pixel-Gaussian associations, enforcing multi-view consistency. The resulting compact yet discriminative feature fields enable stable PnP convergence, achieving state-of-the-art performance on localization benchmarks. Extensive experiments validate that SplitGS-Loc extends the utility of photometric GSFFs to accurate and efficient localization by exploiting Gaussian attributes, without per-scene training or iterative pose refinement.