Semantic-Guided 3D Gaussian Splatting for Transient Object Removal

TL;DR

This paper introduces a semantic-guided method for removing transient objects in 3D Gaussian Splatting reconstructions, improving quality with minimal memory overhead by leveraging vision-language models and semantic classification.

Contribution

It proposes a novel semantic filtering framework that uses CLIP similarity scores for transient removal, resolving parallax ambiguity without high memory costs or motion reliance.

Findings

Improved reconstruction quality on RobustNeRF benchmark

Maintained real-time rendering performance

Validated semantic guidance as effective for transient removal

Abstract

Transient objects in casual multi-view captures cause ghosting artifacts in 3D Gaussian Splatting (3DGS) reconstruction. Existing solutions relied on scene decomposition at significant memory cost or on motion-based heuristics that were vulnerable to parallax ambiguity. A semantic filtering framework was proposed for category-aware transient removal using vision-language models. CLIP similarity scores between rendered views and distractor text prompts were accumulated per-Gaussian across training iterations. Gaussians exceeding a calibrated threshold underwent opacity regularization and periodic pruning. Unlike motion-based approaches, semantic classification resolved parallax ambiguity by identifying object categories independently of motion patterns. Experiments on the RobustNeRF benchmark demonstrated consistent improvement in reconstruction quality over vanilla 3DGS across four sequences, while maintaining minimal memory overhead and real-time rendering performance. Threshold calibration and comparisons with baselines validated semantic guidance as a practical strategy for transient removal in scenarios with predictable distractor categories.