An NCAP-like Safety Indicator for Self-Driving Cars

TL;DR

This paper introduces the Safe-Kamikaze Distance (SKD), a novel safety measure for autonomous cars that evaluates collision avoidance capabilities against adversarial scenarios, demonstrating promising results in simulation.

Contribution

It proposes SKD, a new safety indicator based on trajectory similarity and planning under uncertainty, for assessing autonomous vehicle safety in adversarial scenarios.

Findings

SKD is inversely related to collision probability bounds.

Simulation tests show SKD effectively measures safety in pedestrian crossing scenarios.

Assessment time per test is under 11 seconds, enabling efficient evaluation.

Abstract

This paper proposes a mechanism to assess the safety of autonomous cars. It assesses the car's safety in scenarios where the car must avoid collision with an adversary. Core to this mechanism is a safety measure, called Safe-Kamikaze Distance (SKD), which computes the average similarity between sets of safe adversary's trajectories and kamikaze trajectories close to the safe trajectories. The kamikaze trajectories are generated based on planning under uncertainty techniques, namely the Partially Observable Markov Decision Processes, to account for the partially observed car policy from the point of view of the adversary. We found that SKD is inversely proportional to the upper bound on the probability that a small deformation changes a collision-free trajectory of the adversary into a colliding one. We perform systematic tests on a scenario where the adversary is a pedestrian crossing a single-lane road in front of the car being assessed --which is, one of the scenarios in the Euro-NCAP's Vulnerable Road User (VRU) tests on Autonomous Emergency Braking. Simulation results on assessing cars with basic controllers and a test on a Machine-Learning controller using a high-fidelity simulator indicates promising results for SKD to measure the safety of autonomous cars. Moreover, the time taken for each simulation test is under 11 seconds, enabling a sufficient statistics to compute SKD from simulation to be generated on a quad-core desktop in less than 25 minutes.