Space-Time Forecasting of Dynamic Scenes with Motion-aware Gaussian Grouping

TL;DR

MoGaF is a novel framework that uses motion-aware Gaussian grouping within a 4D Gaussian Splatting representation to improve long-term scene forecasting in dynamic scenes, ensuring physically consistent and stable predictions.

Contribution

Introduces motion-aware Gaussian grouping and group-wise optimization for physically consistent long-term scene extrapolation using 4D Gaussian Splatting.

Findings

Outperforms existing methods in rendering quality and motion plausibility.

Achieves more stable long-term scene forecasting.

Demonstrates effectiveness on synthetic and real-world datasets.

Abstract

Forecasting dynamic scenes remains a fundamental challenge in computer vision, as limited observations make it difficult to capture coherent object-level motion and long-term temporal evolution. We present Motion Group-aware Gaussian Forecasting (MoGaF), a framework for long-term scene extrapolation built upon the 4D Gaussian Splatting representation. MoGaF introduces motion-aware Gaussian grouping and group-wise optimization to enforce physically consistent motion across both rigid and non-rigid regions, yielding spatially coherent dynamic representations. Leveraging this structured space-time representation, a lightweight forecasting module predicts future motion, enabling realistic and temporally stable scene evolution. Experiments on synthetic and real-world datasets demonstrate that MoGaF consistently outperforms existing baselines in rendering quality, motion plausibility, and long-term forecasting stability. Our project page is available at https://slime0519.github.io/mogaf