RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning

TL;DR

RAD introduces a large-scale 3D digital twin and reinforcement learning framework for end-to-end autonomous driving, significantly improving safety and robustness over imitation learning methods by enabling extensive exploration and better handling of out-of-distribution scenarios.

Contribution

The paper presents a novel 3DGS-based RL framework for autonomous driving that integrates imitation learning for better human-like behavior and introduces a new benchmark for evaluation.

Findings

3x lower collision rate compared to IL-based methods

Effective handling of out-of-distribution scenarios

Enhanced safety through specialized reward design

Abstract

Existing end-to-end autonomous driving (AD) algorithms typically follow the Imitation Learning (IL) paradigm, which faces challenges such as causal confusion and an open-loop gap. In this work, we propose RAD, a 3DGS-based closed-loop Reinforcement Learning (RL) framework for end-to-end Autonomous Driving. By leveraging 3DGS techniques, we construct a photorealistic digital replica of the real physical world, enabling the AD policy to extensively explore the state space and learn to handle out-of-distribution scenarios through large-scale trial and error. To enhance safety, we design specialized rewards to guide the policy in effectively responding to safety-critical events and understanding real-world causal relationships. To better align with human driving behavior, we incorporate IL into RL training as a regularization term. We introduce a closed-loop evaluation benchmark consisting of diverse, previously unseen 3DGS environments. Compared to IL-based methods, RAD achieves stronger performance in most closed-loop metrics, particularly exhibiting a 3x lower collision rate. Abundant closed-loop results are presented in the supplementary material. Code is available at https://github.com/hustvl/RAD for facilitating future research.