UniDWM: Towards a Unified Driving World Model via Multifaceted Representation Learning

TL;DR

UniDWM introduces a unified, multifaceted world model for autonomous driving that integrates scene structure, appearance, and dynamics to improve planning, prediction, and reconstruction through a variational autoencoder framework.

Contribution

It proposes a novel unified driving world model that combines structure, appearance, and dynamics in a latent space using a diffusion transformer and VAE principles.

Findings

Enhanced trajectory planning accuracy

Improved 4D scene reconstruction and generation

Effective reasoning across perception, prediction, and planning

Abstract

Achieving reliable and efficient planning in complex driving environments requires a model that can reason over the scene's geometry, appearance, and dynamics. We present UniDWM, a unified driving world model that advances autonomous driving through multifaceted representation learning. UniDWM constructs a structure- and dynamic-aware latent world representation that serves as a physically grounded state space, enabling consistent reasoning across perception, prediction, and planning. Specifically, a joint reconstruction pathway learns to recover the scene's structure, including geometry and visual texture, while a collaborative generation framework leverages a conditional diffusion transformer to forecast future world evolution within the latent space. Furthermore, we show that our UniDWM can be deemed as a variation of VAE, which provides theoretical guidance for the multifaceted representation learning. Extensive experiments demonstrate the effectiveness of UniDWM in trajectory planning, 4D reconstruction and generation, highlighting the potential of multifaceted world representations as a foundation for unified driving intelligence. The code will be publicly available at https://github.com/Say2L/UniDWM.