Xiaomi EV World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving

TL;DR

This paper introduces a unified world model for autonomous driving that combines a high-fidelity 3D scene representation with a causal video generation framework, enhancing simulation and data synthesis capabilities.

Contribution

It presents a novel integrated system, JWM, combining world reconstruction and generation modules for improved stability and fidelity in autonomous driving simulations.

Findings

High-quality online causal video generation in as few as 4 denoising steps

Synergistic integration improves generation stability and cross-frame consistency

Provides a foundation for closed-loop simulation and data synthesis in autonomous driving

Abstract

This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving: world representation and world generation. For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries. WorldRec initializes structured queries in 3D space, leveraging them to aggregate cross-view, cross-temporal features, thereby naturally enforcing spatial consistency across frames and yielding compact yet high-fidelity 3D Gaussian scene representations. For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning through three progressive stages (Teacher Forcing, ODE distillation, and DMD), enabling high-quality online causal video generation in as few as 4 denoising steps. Building on both modules, we further introduce the JWM, which deeply integrates WorldRec and WorldGen to achieve synergistic gains in generation stability, cross-frame consistency, and visual fidelity, providing a solid foundation for closed-loop simulation, data synthesis, and end-to-end training in autonomous driving.