Sparse View Distractor-Free Gaussian Splatting

TL;DR

This paper enhances distractor-free 3D Gaussian Splatting for sparse-view scenarios by integrating prior models like VGGT and VLMs to improve robustness against transient distractors and limited observations.

Contribution

It introduces a novel framework that incorporates geometry and semantic priors to improve distractor-free 3DGS under sparse input conditions.

Findings

Improved robustness in sparse-view 3DGS with transient distractors.

Effective integration of VGGT and VLM priors into existing methods.

Significant performance gains demonstrated through extensive experiments.

Abstract

3D Gaussian Splatting (3DGS) enables efficient training and fast novel view synthesis in static environments. To address challenges posed by transient objects, distractor-free 3DGS methods have emerged and shown promising results when dense image captures are available. However, their performance degrades significantly under sparse input conditions. This limitation primarily stems from the reliance on the color residual heuristics to guide the training, which becomes unreliable with limited observations. In this work, we propose a framework to enhance distractor-free 3DGS under sparse-view conditions by incorporating rich prior information. Specifically, we first adopt the geometry foundation model VGGT to estimate camera parameters and generate a dense set of initial 3D points. Then, we harness the attention maps from VGGT for efficient and accurate semantic entity matching. Additionally, we utilize Vision-Language Models (VLMs) to further identify and preserve the large static regions in the scene. We also demonstrate how these priors can be seamlessly integrated into existing distractor-free 3DGS methods. Extensive experiments confirm the effectiveness and robustness of our approach in mitigating transient distractors for sparse-view 3DGS training.