Tackling Real-World Autonomous Driving using Deep Reinforcement Learning

TL;DR

This paper presents a deep reinforcement learning-based planning system for autonomous vehicles that is trained in simulation and successfully deployed in real-world urban driving, demonstrating good generalization and real-world applicability.

Contribution

The work introduces a model-free deep reinforcement learning planner that predicts acceleration and steering, and a neural network module to replicate vehicle dynamics for sim-to-real transfer.

Findings

System drives smoothly in obstacle-free environments

Effective transfer from simulation to real-world urban driving

Neural network accurately models vehicle dynamics

Abstract

In the typical autonomous driving stack, planning and control systems represent two of the most crucial components in which data retrieved by sensors and processed by perception algorithms are used to implement a safe and comfortable self-driving behavior. In particular, the planning module predicts the path the autonomous car should follow taking the correct high-level maneuver, while control systems perform a sequence of low-level actions, controlling steering angle, throttle and brake. In this work, we propose a model-free Deep Reinforcement Learning Planner training a neural network that predicts both acceleration and steering angle, thus obtaining a single module able to drive the vehicle using the data processed by localization and perception algorithms on board of the self-driving car. In particular, the system that was fully trained in simulation is able to drive smoothly and safely in obstacle-free environments both in simulation and in a real-world urban area of the city of Parma, proving that the system features good generalization capabilities also driving in those parts outside the training scenarios. Moreover, in order to deploy the system on board of the real self-driving car and to reduce the gap between simulated and real-world performances, we also develop a module represented by a tiny neural network able to reproduce the real vehicle dynamic behavior during the training in simulation.