CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving

TL;DR

CIMRL integrates imitation learning with reinforcement learning to develop safer, more effective autonomous driving policies that require less data and handle complex scenarios better than traditional methods.

Contribution

The paper introduces CIMRL, a novel framework that combines imitation and reinforcement learning to improve safety and performance in autonomous driving.

Findings

Achieves state-of-the-art results in simulation benchmarks.

Demonstrates improved safety and robustness in real-world driving.

Reduces data requirements compared to pure imitation learning.

Abstract

Modern approaches to autonomous driving rely heavily on learned components trained with large amounts of human driving data via imitation learning. However, these methods require large amounts of expensive data collection and even then face challenges with safely handling long-tail scenarios and compounding errors over time. At the same time, pure Reinforcement Learning (RL) methods can fail to learn performant policies in sparse, constrained, and challenging-to-define reward settings such as autonomous driving. Both of these challenges make deploying purely cloned or pure RL policies in safety critical applications such as autonomous vehicles challenging. In this paper we propose Combining IMitation and Reinforcement Learning (CIMRL) approach - a safe reinforcement learning framework that enables training driving policies in simulation through leveraging imitative motion priors and safety constraints. CIMRL does not require extensive reward specification and improves on the closed loop behavior of pure cloning methods. By combining RL and imitation, we demonstrate that our method achieves state-of-the-art results in closed loop simulation and real world driving benchmarks.