Towards Better Robustness: Pose-Free 3D Gaussian Splatting for Arbitrarily Long Videos

TL;DR

Rob-GS introduces a robust framework for pose-free 3D Gaussian Splatting that effectively handles long videos by estimating camera poses and segmenting sequences, leading to improved 3D reconstruction quality.

Contribution

The paper presents Rob-GS, a novel method that progressively estimates camera poses and segments long videos for better 3D Gaussian Splatting without known poses.

Findings

Outperforms state-of-the-art methods on multiple datasets.

Effectively handles arbitrarily long video sequences.

Provides stable pose estimation across complex camera trajectories.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a powerful representation due to its efficiency and high-fidelity rendering. 3DGS training requires a known camera pose for each input view, typically obtained by Structure-from-Motion (SfM) pipelines. Pioneering works have attempted to relax this restriction but still face difficulties when handling long sequences with complex camera trajectories. In this paper, we propose Rob-GS, a robust framework to progressively estimate camera poses and optimize 3DGS for arbitrarily long video inputs. In particular, by leveraging the inherent continuity of videos, we design an adjacent pose tracking method to ensure stable pose estimation between consecutive frames. To handle arbitrarily long inputs, we propose a Gaussian visibility retention check strategy to adaptively split the video sequence into several segments and optimize them separately. Extensive experiments on Tanks and Temples, ScanNet, and a self-captured dataset show that Rob-GS outperforms the state-of-the-arts.