DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving

TL;DR

DynVLA introduces a novel Dynamics CoT paradigm for autonomous driving that forecasts compact world dynamics to improve decision-making, outperforming existing Textual and Visual CoT methods in various datasets.

Contribution

The paper presents DynVLA, a new model that predicts compact world dynamics using a Dynamics Tokenizer and decouples ego-centric and environment-centric dynamics for better accuracy.

Findings

DynVLA outperforms Textual CoT and Visual CoT in experiments.

Decoupling ego-centric and environment-centric dynamics improves modeling.

Dynamics Tokenizer enables compact and interpretable world dynamics representation.

Abstract

We propose DynVLA, a driving VLA model that introduces a new CoT paradigm termed Dynamics CoT. DynVLA forecasts compact world dynamics before action generation, enabling more informed and physically grounded decision-making. To obtain compact dynamics representations, DynVLA introduces a Dynamics Tokenizer that compresses future evolution into a small set of dynamics tokens. Considering the rich environment dynamics in interaction-intensive driving scenarios, DynVLA decouples ego-centric and environment-centric dynamics, yielding more accurate world dynamics modeling. We then train DynVLA to generate dynamics tokens before actions through SFT and RFT, improving decision quality while maintaining latency-efficient inference. Compared to Textual CoT, which lacks fine-grained spatiotemporal understanding, and Visual CoT, which introduces substantial redundancy due to dense image prediction, Dynamics CoT captures the evolution of the world in a compact, interpretable, and efficient form. Extensive experiments on NAVSIM, Bench2Drive, and a large-scale in-house dataset demonstrate that DynVLA consistently outperforms Textual CoT and Visual CoT methods, validating the effectiveness and practical value of Dynamics CoT. Project Page: https://yaoyao-jpg.github.io/dynvla.