Improved Robustness and Safety for Autonomous Vehicle Control with Adversarial Reinforcement Learning

TL;DR

This paper explores adversarial reinforcement learning algorithms to enhance the robustness and safety of autonomous vehicle control, demonstrating improved efficiency and reduced collisions in driving scenarios.

Contribution

It introduces a semi-competitive game formulation and compares two algorithms, showing improved performance over traditional methods.

Findings

Enhanced driving efficiency with robust policies

Significant reduction in collision rates

Semi-competitive game formulation captures meaningful disturbances

Abstract

To improve efficiency and reduce failures in autonomous vehicles, research has focused on developing robust and safe learning methods that take into account disturbances in the environment. Existing literature in robust reinforcement learning poses the learning problem as a two player game between the autonomous system and disturbances. This paper examines two different algorithms to solve the game, Robust Adversarial Reinforcement Learning and Neural Fictitious Self Play, and compares performance on an autonomous driving scenario. We extend the game formulation to a semi-competitive setting and demonstrate that the resulting adversary better captures meaningful disturbances that lead to better overall performance. The resulting robust policy exhibits improved driving efficiency while effectively reducing collision rates compared to baseline control policies produced by traditional reinforcement learning methods.