Learning to drive via Apprenticeship Learning and Deep Reinforcement Learning

TL;DR

This paper presents a novel approach combining apprenticeship learning and deep reinforcement learning to enable autonomous vehicles to learn human-like driving behaviors with continuous actions, addressing real-world driving complexities.

Contribution

It introduces a method that integrates gradient inverse reinforcement learning with deep RL algorithms to learn driving behaviors adaptable to individual styles.

Findings

Agent performs human-like driving in simulation

Method outperforms baseline in safety and comfort metrics

Learns personalized driving styles effectively

Abstract

With the implementation of reinforcement learning (RL) algorithms, current state-of-art autonomous vehicle technology have the potential to get closer to full automation. However, most of the applications have been limited to game domains or discrete action space which are far from the real world driving. Moreover, it is very tough to tune the parameters of reward mechanism since the driving styles vary a lot among the different users. For instance, an aggressive driver may prefer driving with high acceleration whereas some conservative drivers prefer a safer driving style. Therefore, we propose an apprenticeship learning in combination with deep reinforcement learning approach that allows the agent to learn the driving and stopping behaviors with continuous actions. We use gradient inverse reinforcement learning (GIRL) algorithm to recover the unknown reward function and employ REINFORCE as well as Deep Deterministic Policy Gradient algorithm (DDPG) to learn the optimal policy. The performance of our method is evaluated in simulation-based scenario and the results demonstrate that the agent performs human like driving and even better in some aspects after training.