Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)

TL;DR

Raw2Drive introduces a novel reinforcement learning framework for end-to-end autonomous driving that effectively utilizes aligned world models trained on privileged and raw sensor data, achieving state-of-the-art results in CARLA benchmarks.

Contribution

The paper presents Raw2Drive, a dual-stream model-based RL approach that bridges privileged and raw sensor data for autonomous driving, a novel method in this domain.

Findings

Achieves state-of-the-art performance on CARLA Leaderboard 2.0

First RL-based end-to-end autonomous driving method on CARLA benchmarks

Effectively aligns privileged and raw sensor world models during training

Abstract

Reinforcement Learning (RL) can mitigate the causal confusion and distribution shift inherent to imitation learning (IL). However, applying RL to end-to-end autonomous driving (E2E-AD) remains an open problem for its training difficulty, and IL is still the mainstream paradigm in both academia and industry. Recently Model-based Reinforcement Learning (MBRL) have demonstrated promising results in neural planning; however, these methods typically require privileged information as input rather than raw sensor data. We fill this gap by designing Raw2Drive, a dual-stream MBRL approach. Initially, we efficiently train an auxiliary privileged world model paired with a neural planner that uses privileged information as input. Subsequently, we introduce a raw sensor world model trained via our proposed Guidance Mechanism, which ensures consistency between the raw sensor world model and the privileged world model during rollouts. Finally, the raw sensor world model combines the prior knowledge embedded in the heads of the privileged world model to effectively guide the training of the raw sensor policy. Raw2Drive is so far the only RL based end-to-end method on CARLA Leaderboard 2.0, and Bench2Drive and it achieves state-of-the-art performance.