VectorWorld: Efficient Streaming World Model via Diffusion Flow on Vector Graphs

TL;DR

VectorWorld is a novel streaming world model for autonomous driving that incrementally generates lane-agent vector-graph tiles, enabling real-time, stable, long-horizon closed-loop simulations with improved fidelity and initialization compatibility.

Contribution

It introduces a new incremental, vector-graph-based world model with motion-aware initialization and solver-free outpainting for real-time autonomous driving simulation.

Findings

Supports stable, real-time 1 km+ closed-loop rollouts

Improves map-structure fidelity and initialization validity

Enables long-horizon simulation with stable dynamics

Abstract

Closed-loop evaluation of autonomous-driving policies requires interactive simulation beyond log replay. However, existing generative world models often degrade in closed loop due to (i) history-free initialization that mismatches policy inputs, (ii) multi-step sampling latency that violates real-time budgets, and (iii) compounding kinematic infeasibility over long horizons. We propose VectorWorld, a streaming world model that incrementally generates ego-centric $64 \mathrm{m}\times 64\mathrm{m}$ lane--agent vector-graph tiles during rollout. VectorWorld aligns initialization with history-conditioned policies by producing a policy-compatible interaction state via a motion-aware gated VAE. It enables real-time outpainting via solver-free one-step masked completion with an edge-gated relational DiT trained with interval-conditioned MeanFlow and JVP-based large-step supervision. To stabilize long-horizon rollouts, we introduce $\Delta$Sim, a physics-aligned non-ego (NPC) policy with hybrid discrete--continuous actions and differentiable kinematic logit shaping. On Waymo open motion and nuPlan, VectorWorld improves map-structure fidelity and initialization validity, and supports stable, real-time $1\mathrm{km}+$ closed-loop rollouts (\href{https://github.com/jiangchaokang/VectorWorld}{code}).