GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning

TL;DR

GEM introduces a continuous-time Gaussian primitives approach for more flexible, accurate, and interpretable 3D occupancy forecasting and motion planning in autonomous driving.

Contribution

It proposes a non-autoregressive, continuous-time Gaussian evolution model that improves scene prediction and planning by explicitly modeling scene dynamics as Gaussian primitives.

Findings

Achieves state-of-the-art occupancy forecasting accuracy.

Provides strong motion planning performance.

Enables flexible temporal querying of scene evolution.

Abstract

Future 3D semantic occupancy forecasting and motion planning are central to autonomous driving, as they require models to reason about how surrounding scenes evolve and how the ego vehicle should act. Existing occupancy world models commonly discretize scenes into latent embeddings, volumetric features, or quantized tokens, and forecast future states through fixed-step autoregressive generation. This limits temporal flexibility, obscures scene evolution, accumulates errors over long horizons, and poorly matches the continuous-time dynamics of real driving scenes. We propose GEM, a Gaussian Evolution Model for non-autoregressive occupancy world modeling, where driving scenes are represented as explicit continuous 4D Gaussian primitives with learned dynamics. Instead of rolling out future occupancy states step by step, GEM directly queries the Gaussian world representation at arbitrary timestamps and splats the corresponding conditional 3D Gaussians into semantic occupancy volumes. This enables efficient forecasting over the full horizon while retaining a compact and interpretable scene representation. By decoupling spatial geometry, temporal support, and primitive motion, GEM makes the predicted world easier to inspect, as each primitive's evolution can be followed continuously over time. The same representation also supports motion planning by predicting future ego trajectories from the learned Gaussian world. Extensive experiments show that GEM achieves state-of-the-art future semantic occupancy forecasting and strong motion planning performance, while providing flexible temporal querying.