T-3DGS: Removing Transient Objects for 3D Scene Reconstruction

TL;DR

T-3DGS introduces a robust framework that effectively filters out transient objects in video sequences, significantly improving the quality of 3D scene reconstructions using Gaussian Splatting.

Contribution

The paper presents a novel two-step method combining unsupervised classification and segmentation with tracking to remove transient distractors during 3D reconstruction.

Findings

Outperforms state-of-the-art methods on various datasets.

Enables high-fidelity 3D reconstructions in challenging scenarios.

Effectively distinguishes transient objects from static scene elements.

Abstract

Transient objects in video sequences can significantly degrade the quality of 3D scene reconstructions. To address this challenge, we propose T-3DGS, a novel framework that robustly filters out transient distractors during 3D reconstruction using Gaussian Splatting. Our framework consists of two steps. First, we employ an unsupervised classification network that distinguishes transient objects from static scene elements by leveraging their distinct training dynamics within the reconstruction process. Second, we refine these initial detections by integrating an off-the-shelf segmentation method with a bidirectional tracking module, which together enhance boundary accuracy and temporal coherence. Evaluations on both sparsely and densely captured video datasets demonstrate that T-3DGS significantly outperforms state-of-the-art approaches, enabling high-fidelity 3D reconstructions in challenging, real-world scenarios.