Camera Pose Refinement via 3D Gaussian Splatting

TL;DR

This paper introduces GS-SMC, a novel camera pose refinement method using 3D Gaussian Splatting that improves accuracy without scene reprocessing or retraining, leveraging existing 3D models and geometric constraints.

Contribution

The proposed framework enables efficient, training-free camera pose refinement by integrating 3D Gaussian Splatting with epipolar geometry, outperforming existing methods on standard datasets.

Findings

Achieves over 50% reduction in median translation and rotation errors on 7-Scenes.

Outperforms state-of-the-art methods on Cambridge Landmarks dataset.

Employs existing 3D models for lightweight, versatile pose refinement.

Abstract

Camera pose refinement aims at improving the accuracy of initial pose estimation for applications in 3D computer vision. Most refinement approaches rely on 2D-3D correspondences with specific descriptors or dedicated networks, requiring reconstructing the scene again for a different descriptor or fully retraining the network for each scene. Some recent methods instead infer pose from feature similarity, but their lack of geometry constraints results in less accuracy. To overcome these limitations, we propose a novel camera pose refinement framework leveraging 3D Gaussian Splatting (3DGS), referred to as GS-SMC. Given the widespread usage of 3DGS, our method can employ an existing 3DGS model to render novel views, providing a lightweight solution that can be directly applied to diverse scenes without additional training or fine-tuning. Specifically, we introduce an iterative optimization approach, which refines the camera pose using epipolar geometric constraints among the query and multiple rendered images. Our method allows flexibly choosing feature extractors and matchers to establish these constraints. Extensive empirical evaluations on the 7-Scenes and the Cambridge Landmarks datasets demonstrate that our method outperforms state-of-the-art camera pose refinement approaches, achieving 53.3% and 56.9% reductions in median translation and rotation errors on 7-Scenes, and 40.7% and 53.2% on Cambridge.