ODE-GS: Latent ODEs for Dynamic Scene Extrapolation with 3D Gaussian Splatting

TL;DR

ODE-GS introduces a continuous-time latent dynamics model combining Gaussian Splatting and neural ODEs for accurate future extrapolation of dynamic 3D scenes, outperforming existing methods.

Contribution

It presents a novel framework that models Gaussian parameter trajectories as continuous-time latent dynamics, enabling timestamp-independent scene extrapolation.

Findings

Achieves 19.8% improvement on extrapolation metrics over baselines.

Produces smooth, physically plausible future scene trajectories.

Outperforms existing methods on D-NeRF, NVFi, and HyperNeRF benchmarks.

Abstract

We introduce ODE-GS, a novel approach that integrates 3D Gaussian Splatting with latent neural ordinary differential equations (ODEs) to enable future extrapolation of dynamic 3D scenes. Unlike existing dynamic scene reconstruction methods, which rely on time-conditioned deformation networks and are limited to interpolation within a fixed time window, ODE-GS eliminates timestamp dependency by modeling Gaussian parameter trajectories as continuous-time latent dynamics. Our approach first learns an interpolation model to generate accurate Gaussian trajectories within the observed window, then trains a Transformer encoder to aggregate past trajectories into a latent state evolved via a neural ODE. Finally, numerical integration produces smooth, physically plausible future Gaussian trajectories, enabling rendering at arbitrary future timestamps. On the D-NeRF, NVFi, and HyperNeRF benchmarks, ODE-GS achieves state-of-the-art extrapolation performance, improving metrics by 19.8% compared to leading baselines, demonstrating its ability to accurately represent and predict 3D scene dynamics.