Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validation

TL;DR

This paper enhances Adaptive Stress Testing (AST) for autonomous vehicle validation by incorporating domain knowledge through reward augmentation, enabling discovery of a broader range of failure scenarios in simulation.

Contribution

The work introduces a reward augmentation technique into AST, improving its ability to find diverse and meaningful failure scenarios for autonomous vehicles.

Findings

Enhanced AST discovers a larger failure space.

The method identifies more diverse failure scenarios.

Improved validation of autonomous vehicle policies.

Abstract

Determining possible failure scenarios is a critical step in the evaluation of autonomous vehicle systems. Real-world vehicle testing is commonly employed for autonomous vehicle validation, but the costs and time requirements are high. Consequently, simulation-driven methods such as Adaptive Stress Testing (AST) have been proposed to aid in validation. AST formulates the problem of finding the most likely failure scenarios as a Markov decision process, which can be solved using reinforcement learning. In practice, AST tends to find scenarios where failure is unavoidable and tends to repeatedly discover the same types of failures of a system. This work addresses these issues by encoding domain relevant information into the search procedure. With this modification, the AST method discovers a larger and more expressive subset of the failure space when compared to the original AST formulation. We show that our approach is able to identify useful failure scenarios of an autonomous vehicle policy.