RoDyGS: Robust Dynamic Gaussian Splatting for Casual Videos

TL;DR

RoDyGS is a novel method for 4D reconstruction from casual monocular videos that explicitly separates static and dynamic elements, applying spatiotemporal regularization for physically plausible and consistent scene modeling.

Contribution

It introduces RoDyGS, a new approach for dynamic scene reconstruction from monocular videos, and a comprehensive benchmark, Kubric-MRig, for evaluating such methods.

Findings

RoDyGS outperforms previous pose-free dynamic view synthesis methods.

Achieves competitive quality with static view synthesis approaches.

Demonstrates effective separation of static and dynamic scene components.

Abstract

4D reconstruction from casually captured monocular videos is challenging due to inherent ambiguity in reconstructing dynamic 3D geometry. To address this challenge, we introduce Robust Dynamic Gaussian Splatting (RoDyGS), a method that reconstructs dynamic scene representation from casual monocular videos. RoDyGS explicitly separates static and dynamic scene elements, and applies spatiotemporal regularization to enforce physically plausible geometry and temporally consistent motion. Furthermore, we propose a comprehensive benchmark, Kubric-MRig, which provides extensive camera and object motion along with simultaneous multi-view capture, features that are absent in previous benchmarks. Experiments demonstrate that RoDyGS significantly outperforms previous pose-free dynamic novel view synthesis approaches and achieves competitive rendering quality compared to existing pose-free static novel view synthesis approaches. Our proejct page is available at https://rodygs.github.io