DGGT: Feedforward 4D Reconstruction of Dynamic Driving Scenes using Unposed Images

TL;DR

DGGT is a fast, pose-free 4D scene reconstruction method for dynamic driving scenes that predicts camera poses and 3D Gaussian maps directly from unposed images, enabling scalable and high-quality reconstructions.

Contribution

The paper introduces DGGT, a novel feedforward framework that jointly predicts camera poses and 3D scene representations from unposed images, eliminating the need for per-scene optimization or known calibration.

Findings

Outperforms prior methods on large-scale driving benchmarks.

Supports arbitrary number of views and long sequences.

Achieves state-of-the-art speed and quality in 4D reconstruction.

Abstract

Autonomous driving needs fast, scalable 4D reconstruction and re-simulation for training and evaluation, yet most methods for dynamic driving scenes still rely on per-scene optimization, known camera calibration, or short frame windows, making them slow and impractical. We revisit this problem from a feedforward perspective and introduce \textbf{Driving Gaussian Grounded Transformer (DGGT)}, a unified framework for pose-free dynamic scene reconstruction. We note that the existing formulations, treating camera pose as a required input, limit flexibility and scalability. Instead, we reformulate pose as an output of the model, enabling reconstruction directly from sparse, unposed images and supporting an arbitrary number of views for long sequences. Our approach jointly predicts per-frame 3D Gaussian maps and camera parameters, disentangles dynamics with a lightweight dynamic head, and preserves temporal consistency with a lifespan head that modulates visibility over time. A diffusion-based rendering refinement further reduces motion/interpolation artifacts and improves novel-view quality under sparse inputs. The result is a single-pass, pose-free algorithm that achieves state-of-the-art performance and speed. Trained and evaluated on large-scale driving benchmarks (Waymo, nuScenes, Argoverse2), our method outperforms prior work both when trained on each dataset and in zero-shot transfer across datasets, and it scales well as the number of input frames increases.