Towards Guaranteed Safety Assurance of Automated Driving Systems with Scenario Sampling: An Invariant Set Perspective (Extended Version)

TL;DR

This paper introduces a set invariance approach to determine the sufficient number of scenarios for validating the safety of automated driving systems, providing probabilistic solutions and algorithms with theoretical guarantees.

Contribution

It formulates the scenario sampling safety assurance problem as a set invariance validation task and offers novel probabilistic algorithms with finite-sampling analysis for safety validation and quantification.

Findings

Proposes a set invariance perspective for safety validation.

Provides probabilistic complete and asymptotically optimal algorithms.

Demonstrates effectiveness through simulation experiments.

Abstract

How many scenarios are sufficient to validate the safe Operational Design Domain (ODD) of an Automated Driving System (ADS) equipped vehicle? Is a more significant number of sampled scenarios guaranteeing a more accurate safety assessment of the ADS? Despite the various empirical success of ADS safety evaluation with scenario sampling in practice, some of the fundamental properties are largely unknown. This paper seeks to remedy this gap by formulating and tackling the scenario sampling safety assurance problem from a set invariance perspective. First, a novel conceptual equivalence is drawn between the scenario sampling safety assurance problem and the data-driven robustly controlled forward invariant set validation and quantification problem. This paper then provides a series of resolution complete and probabilistic complete solutions with finite-sampling analyses for the safety validation problem that authenticates a given ODD. On the other hand, the quantification problem escalates the validation challenge and starts looking for a safe sub-domain of a particular property. This inspires various algorithms that are provably probabilistic incomplete, probabilistic complete but sub-optimal, and asymptotically optimal. Finally, the proposed asymptotically optimal scenario sampling safety quantification algorithm is also empirically demonstrated through simulation experiments.