Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout

TL;DR

Driver-WM is a novel driver-centric latent world model that predicts in-cabin driver dynamics conditioned on external traffic context, enhancing safety and interpretability in autonomous driving systems.

Contribution

It introduces a causal, dual-stream architecture that unifies physical and semantic driver modeling with external traffic, enabling long-term in-cabin dynamics prediction.

Findings

Improves long-horizon geometric forecasting for high-motion maneuvers.

Enhances semantic alignment of driver and traffic states.

Enables controlled interventions for mechanism analysis.

Abstract

Safe L2/L3 driving automation requires anticipating human-in-the-loop reactions during shared-control transitions. While most driving world models forecast the external environment, in-cabin intelligence remains strictly recognition-oriented and lacks multi-step rollout capabilities for driver dynamics. We introduce Driver-WM, a driver-centric latent world model that rolls out in-cabin dynamics causally conditioned on out-cabin traffic context. This formulation unifies physical kinematics forecasting with auxiliary behavioral and emotional semantic recognition. Operating in a compact latent space constructed from frozen vision-language features, Driver-WM adopts a dual-stream architecture to separately encode external traffic and internal driver states. These streams are directionally coupled via a gated causal injection mechanism, which uses a learned vector gate to modulate external contextual perturbations while strictly enforcing temporal causality. Evaluations on a multi-task assistive driving benchmark demonstrate that Driver-WM yields robust long-horizon geometric forecasting for reactive high-motion maneuvers and improves semantic alignment for both driver and traffic states. Finally, the explicit external-to-internal conditioning allows for controlled test-time interventions to systematically analyze mechanism responses.