RobustSplat++: Decoupling Densification, Dynamics, and Illumination for In-the-Wild 3DGS

TL;DR

RobustSplat++ improves in-the-wild 3D scene rendering by decoupling densification, dynamics, and illumination, effectively handling transient objects and lighting variations for more robust and realistic results.

Contribution

It introduces a delayed Gaussian growth strategy and a scale-cascaded mask bootstrapping approach to enhance robustness against transient disturbances and illumination changes in 3D Gaussian Splatting.

Findings

Outperforms existing methods on multiple challenging datasets.

Effectively handles transient objects and illumination variations.

Demonstrates robustness and improved scene detail capture.

Abstract

3D Gaussian Splatting (3DGS) has gained significant attention for its real-time, photo-realistic rendering in novel-view synthesis and 3D modeling. However, existing methods struggle with accurately modeling in-the-wild scenes affected by transient objects and illuminations, leading to artifacts in the rendered images. We identify that the Gaussian densification process, while enhancing scene detail capture, unintentionally contributes to these artifacts by growing additional Gaussians that model transient disturbances and illumination variations. To address this, we propose RobustSplat++, a robust solution based on several critical designs. First, we introduce a delayed Gaussian growth strategy that prioritizes optimizing static scene structure before allowing Gaussian splitting/cloning, mitigating overfitting to transient objects in early optimization. Second, we design a scale-cascaded mask bootstrapping approach that first leverages lower-resolution feature similarity supervision for reliable initial transient mask estimation, taking advantage of its stronger semantic consistency and robustness to noise, and then progresses to high-resolution supervision to achieve more precise mask prediction. Third, we incorporate the delayed Gaussian growth strategy and mask bootstrapping with appearance modeling to handling in-the-wild scenes including transients and illuminations. Extensive experiments on multiple challenging datasets show that our method outperforms existing methods, clearly demonstrating the robustness and effectiveness of our method.