Comparing Normalizing Flows with Kernel Density Estimation in Estimating Risk of Automated Driving Systems

TL;DR

This paper compares Normalizing Flows and Kernel Density Estimation for estimating the probability distribution of parameters in safety validation of Automated Driving Systems, highlighting NF's advantages in high-dimensional scenarios.

Contribution

It demonstrates the effectiveness of Normalizing Flows in estimating complex PDFs for risk assessment, outperforming KDE especially in high-dimensional spaces.

Findings

NF provides more accurate risk uncertainty estimation.

NF is less sensitive to the curse of dimensionality.

NF requires more computational resources than KDE.

Abstract

The development of safety validation methods is essential for the safe deployment and operation of Automated Driving Systems (ADSs). One of the goals of safety validation is to prospectively evaluate the risk of an ADS dealing with real-world traffic. Scenario-based assessment is a widely-used approach, where test cases are derived from real-world driving data. To allow for a quantitative analysis of the system performance, the exposure of the scenarios must be accurately estimated. The exposure of scenarios at parameter level is expressed using a Probability Density Function (PDF). However, assumptions about the PDF, such as parameter independence, can introduce errors, while avoiding assumptions often leads to oversimplified models with limited parameters to mitigate the curse of dimensionality. This paper considers the use of Normalizing Flows (NF) for estimating the PDF of the parameters. NF are a class of generative models that transform a simple base distribution into a complex one using a sequence of invertible and differentiable mappings, enabling flexible, high-dimensional density estimation without restrictive assumptions on the PDF's shape. We demonstrate the effectiveness of NF in quantifying risk and risk uncertainty of an ADS, comparing its performance with Kernel Density Estimation (KDE), a traditional method for non-parametric PDF estimation. While NF require more computational resources compared to KDE, NF is less sensitive to the curse of dimensionality. As a result, NF can improve risk uncertainty estimation, offering a more precise assessment of an ADS's safety. This work illustrates the potential of NF in scenario-based safety. Future work involves experimenting more with using NF for scenario generation and optimizing the NF architecture, transformation types, and training hyperparameters to further enhance their applicability.