VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning

TL;DR

VADv2 introduces a probabilistic planning approach for end-to-end autonomous driving, modeling uncertainty with a probabilistic field function and tokenized planning vocabulary, achieving state-of-the-art results.

Contribution

It proposes a novel probabilistic planning model that discretizes and tokenizes the action space, enabling effective uncertainty modeling in autonomous driving.

Findings

Achieves state-of-the-art performance on CARLA Town05 benchmark.

Outperforms existing methods on Bench2Drive benchmark.

Demonstrates effectiveness in real-world applications through comprehensive evaluations.

Abstract

Learning a human-like driving policy from large-scale driving demonstrations is promising, but the uncertainty and non-deterministic nature of planning make it challenging. Existing learning-based planning methods follow a deterministic paradigm to directly regress the action, failing to cope with the uncertainty problem. In this work, we propose a probabilistic planning model for end-to-end autonomous driving, termed VADv2. We resort to a probabilistic field function to model the mapping from the action space to the probabilistic distribution. Since the planning action space is a high-dimensional continuous spatiotemporal space and hard to tackle, we first discretize the planning action space to a large planning vocabulary and then tokenize the planning vocabulary into planning tokens. Planning tokens interact with scene tokens and output the probabilistic distribution of action. Mass driving demonstrations are leveraged to supervise the distribution. VADv2 achieves state-of-the-art closed-loop performance on the CARLA Town05 benchmark, significantly outperforming existing methods, and also leads the recent Bench2Drive benchmark. We further provide comprehensive evaluations on NAVSIM and a large-scale 3DGS-based benchmark, demonstrating its effectiveness in real-world applications. Code is available at https://github.com/hustvl/VAD.