Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving

TL;DR

This paper introduces a multi-agent reinforcement learning approach to generate failure scenarios for rule-based agents, particularly in autonomous driving, to improve safety testing by efficiently identifying critical failure cases.

Contribution

It proposes a novel adversarial reinforcement learning method to find failure scenarios in multi-agent systems, addressing the challenge of testing complex autonomous driving algorithms.

Findings

Effective failure scenario generation demonstrated in simulations

Improved safety testing for autonomous driving systems

Method outperforms traditional testing approaches

Abstract

We examine the problem of adversarial reinforcement learning for multi-agent domains including a rule-based agent. Rule-based algorithms are required in safety-critical applications for them to work properly in a wide range of situations. Hence, every effort is made to find failure scenarios during the development phase. However, as the software becomes complicated, finding failure cases becomes difficult. Especially in multi-agent domains, such as autonomous driving environments, it is much harder to find useful failure scenarios that help us improve the algorithm. We propose a method for efficiently finding failure scenarios; this method trains the adversarial agents using multi-agent reinforcement learning such that the tested rule-based agent fails. We demonstrate the effectiveness of our proposed method using a simple environment and autonomous driving simulator.