Rethinking Pose Refinement in 3D Gaussian Splatting under Pose Prior and Geometric Uncertainty

TL;DR

This paper enhances 3D Gaussian Splatting-based pose refinement by explicitly modeling pose and geometric uncertainties, leading to more robust and accurate visual localization without retraining.

Contribution

It introduces a relocalization framework combining Monte Carlo sampling with Fisher Information-based PnP to handle uncertainties in 3DGS pose refinement.

Findings

Improves localization accuracy across indoor and outdoor benchmarks.

Increases stability under pose and depth noise.

Requires no retraining or additional supervision.

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a powerful scene representation and is increasingly used for visual localization and pose refinement. However, despite its high-quality differentiable rendering, the robustness of 3DGS-based pose refinement remains highly sensitive to both the initial camera pose and the reconstructed geometry. In this work, we take a closer look at these limitations and identify two major sources of uncertainty: (i) pose prior uncertainty, which often arises from regression or retrieval models that output a single deterministic estimate, and (ii) geometric uncertainty, caused by imperfections in the 3DGS reconstruction that propagate errors into PnP solvers. Such uncertainties can distort reprojection geometry and destabilize optimization, even when the rendered appearance still looks plausible. To address these uncertainties, we introduce a relocalization framework that combines Monte Carlo pose sampling with Fisher Information-based PnP optimization. Our method explicitly accounts for both pose and geometric uncertainty and requires no retraining or additional supervision. Across diverse indoor and outdoor benchmarks, our approach consistently improves localization accuracy and significantly increases stability under pose and depth noise.