STDR: Spatio-Temporal Decoupling for Real-Time Dynamic Scene Rendering

TL;DR

This paper introduces STDR, a module that decouples spatial and temporal information in 3D Gaussian Splatting, significantly improving real-time dynamic scene reconstruction quality and consistency.

Contribution

The paper proposes a novel spatio-temporal decoupling module for 3D Gaussian Splatting, enhancing dynamic scene reconstruction by disentangling spatial and temporal features.

Findings

Improved spatio-temporal coherence in dynamic scene rendering.

Enhanced reconstruction quality on synthetic and real-world data.

Compatible with existing 3D Gaussian Splatting frameworks.

Abstract

Although dynamic scene reconstruction has long been a fundamental challenge in 3D vision, the recent emergence of 3D Gaussian Splatting (3DGS) offers a promising direction by enabling high-quality, real-time rendering through explicit Gaussian primitives. However, existing 3DGS-based methods for dynamic reconstruction often suffer from \textit{spatio-temporal incoherence} during initialization, where canonical Gaussians are constructed by aggregating observations from multiple frames without temporal distinction. This results in spatio-temporally entangled representations, making it difficult to model dynamic motion accurately. To overcome this limitation, we propose \textbf{STDR} (Spatio-Temporal Decoupling for Real-time rendering), a plug-and-play module that learns spatio-temporal probability distributions for each Gaussian. STDR introduces a spatio-temporal mask, a separated deformation field, and a consistency regularization to jointly disentangle spatial and temporal patterns. Extensive experiments demonstrate that incorporating our module into existing 3DGS-based dynamic scene reconstruction frameworks leads to notable improvements in both reconstruction quality and spatio-temporal consistency across synthetic and real-world benchmarks.