A Counterfactual Safety Margin Perspective on the Scoring of Autonomous Vehicles' Riskiness

TL;DR

This paper introduces a counterfactual safety margin framework for evaluating autonomous vehicles' riskiness, enabling risk assessment even with limited data and undisclosed policies, aiding regulators and insurers.

Contribution

It proposes a novel counterfactual safety margin concept and a data-driven risk assessment method applicable without knowing AV behavioral policies.

Findings

Safety margin correlates with driving policy quality

Critical scenarios identified through counterfactual simulations

Method applicable for regulatory and insurance risk evaluation

Abstract

Autonomous Vehicles (AVs) promise a range of societal advantages, including broader access to mobility, reduced road accidents, and enhanced transportation efficiency. However, evaluating the risks linked to AVs is complex due to limited historical data and the swift progression of technology. This paper presents a data-driven framework for assessing the risk of different AVs' behaviors in various operational design domains (ODDs), based on counterfactual simulations of "misbehaving" road users. We propose the notion of counterfactual safety margin, which represents the minimum deviation from nominal behavior that could cause a collision. This methodology not only pinpoints the most critical scenarios but also quantifies the (relative) risk's frequency and severity concerning AVs. Importantly, we show that our approach is applicable even when the AV's behavioral policy remains undisclosed, through worst- and best-case analyses, benefiting external entities like regulators and risk evaluators. Our experimental outcomes demonstrate the correlation between the safety margin, the quality of the driving policy, and the ODD, shedding light on the relative risks of different AV providers. Overall, this work contributes to the safety assessment of AVs and addresses legislative and insurance concerns surrounding this burgeoning technology.