DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes

TL;DR

DeSiRe-GS is a self-supervised Gaussian splatting method that decomposes static and dynamic elements in urban scenes, enabling high-fidelity surface reconstruction for autonomous driving with improved accuracy and robustness.

Contribution

The paper introduces a novel two-stage optimization pipeline for dynamic street Gaussian splatting, addressing data sparsity and overfitting in urban scene reconstruction.

Findings

Outperforms prior self-supervised methods in accuracy

Achieves surface reconstruction quality comparable to annotation-based methods

Effectively decomposes static and dynamic scene components

Abstract

We present DeSiRe-GS, a self-supervised gaussian splatting representation, enabling effective static-dynamic decomposition and high-fidelity surface reconstruction in complex driving scenarios. Our approach employs a two-stage optimization pipeline of dynamic street Gaussians. In the first stage, we extract 2D motion masks based on the observation that 3D Gaussian Splatting inherently can reconstruct only the static regions in dynamic environments. These extracted 2D motion priors are then mapped into the Gaussian space in a differentiable manner, leveraging an efficient formulation of dynamic Gaussians in the second stage. Combined with the introduced geometric regularizations, our method are able to address the over-fitting issues caused by data sparsity in autonomous driving, reconstructing physically plausible Gaussians that align with object surfaces rather than floating in air. Furthermore, we introduce temporal cross-view consistency to ensure coherence across time and viewpoints, resulting in high-quality surface reconstruction. Comprehensive experiments demonstrate the efficiency and effectiveness of DeSiRe-GS, surpassing prior self-supervised arts and achieving accuracy comparable to methods relying on external 3D bounding box annotations. Code is available at https://github.com/chengweialan/DeSiRe-GS