Deep Reinforcement Learning for Autonomous Driving

TL;DR

This paper applies deep reinforcement learning, specifically DDPG, to autonomous driving in simulation, addressing complex state and action spaces while ensuring safety, and demonstrates promising results in TORCS environment.

Contribution

The paper adapts DDPG for autonomous driving in simulation, designing specific network architectures and reward functions for complex environments.

Findings

Effective in TORCS simulation environments

Handles continuous state and action spaces

Shows promising quantitative and qualitative results

Abstract

Reinforcement learning has steadily improved and outperform human in lots of traditional games since the resurgence of deep neural network. However, these success is not easy to be copied to autonomous driving because the state spaces in real world are extreme complex and action spaces are continuous and fine control is required. Moreover, the autonomous driving vehicles must also keep functional safety under the complex environments. To deal with these challenges, we first adopt the deep deterministic policy gradient (DDPG) algorithm, which has the capacity to handle complex state and action spaces in continuous domain. We then choose The Open Racing Car Simulator (TORCS) as our environment to avoid physical damage. Meanwhile, we select a set of appropriate sensor information from TORCS and design our own rewarder. In order to fit DDPG algorithm to TORCS, we design our network architecture for both actor and critic inside DDPG paradigm. To demonstrate the effectiveness of our model, We evaluate on different modes in TORCS and show both quantitative and qualitative results.