Multi-Vehicle Mixed-Reality Reinforcement Learning for Autonomous Multi-Lane Driving

TL;DR

This paper introduces a safe, mixed-reality reinforcement learning framework for autonomous multi-lane driving that enables real-world policy adaptation while minimizing collisions through virtual simulation of interactions.

Contribution

It presents a novel sim2real approach using mixed-reality to safely learn multi-vehicle driving policies with real-time adaptation and collision simulation.

Findings

Collisions are significantly reduced after few mixed-reality training runs.

The framework enables safe policy learning in shared multi-vehicle environments.

Real-world online adaptation improves driving policy safety and efficiency.

Abstract

Autonomous driving promises to transform road transport. Multi-vehicle and multi-lane scenarios, however, present unique challenges due to constrained navigation and unpredictable vehicle interactions. Learning-based methods---such as deep reinforcement learning---are emerging as a promising approach to automatically design intelligent driving policies that can cope with these challenges. Yet, the process of safely learning multi-vehicle driving behaviours is hard: while collisions---and their near-avoidance---are essential to the learning process, directly executing immature policies on autonomous vehicles raises considerable safety concerns. In this article, we present a safe and efficient framework that enables the learning of driving policies for autonomous vehicles operating in a shared workspace, where the absence of collisions cannot be guaranteed. Key to our learning procedure is a sim2real approach that uses real-world online policy adaptation in a mixed-reality setup, where other vehicles and static obstacles exist in the virtual domain. This allows us to perform safe learning by simulating (and learning from) collisions between the learning agent(s) and other objects in virtual reality. Our results demonstrate that, after only a few runs in mixed-reality, collisions are significantly reduced.