Efficient falsification approach for autonomous vehicle validation using a parameter optimisation technique based on reinforcement learning

TL;DR

This paper introduces an efficient reinforcement learning-based method to generate challenging scenarios for autonomous vehicle testing, improving safety validation by optimizing the search for failure-inducing cases.

Contribution

It proposes a novel reinforcement learning approach for scenario optimization, enhancing the efficiency of falsification in autonomous vehicle validation.

Findings

More efficient search for failure scenarios

Improved safety validation process

Effective use of reinforcement learning in scenario generation

Abstract

The widescale deployment of Autonomous Vehicles (AV) appears to be imminent despite many safety challenges that are yet to be resolved. It is well-known that there are no universally agreed Verification and Validation (VV) methodologies guarantee absolute safety, which is crucial for the acceptance of this technology. The uncertainties in the behaviour of the traffic participants and the dynamic world cause stochastic reactions in advanced autonomous systems. The addition of ML algorithms and probabilistic techniques adds significant complexity to the process for real-world testing when compared to traditional methods. Most research in this area focuses on generating challenging concrete scenarios or test cases to evaluate the system performance by looking at the frequency distribution of extracted parameters as collected from the real-world data. These approaches generally employ Monte-Carlo simulation and importance sampling to generate critical cases. This paper presents an efficient falsification method to evaluate the System Under Test. The approach is based on a parameter optimisation problem to search for challenging scenarios. The optimisation process aims at finding the challenging case that has maximum return. The method applies policy-gradient reinforcement learning algorithm to enable the learning. The riskiness of the scenario is measured by the well established RSS safety metric, euclidean distance, and instance of a collision. We demonstrate that by using the proposed method, we can more efficiently search for challenging scenarios which could cause the system to fail in order to satisfy the safety requirements.