Failure Probability Estimation for Black-Box Autonomous Systems using State-Dependent Importance Sampling Proposals

TL;DR

This paper introduces an adaptive importance sampling method for accurately estimating failure probabilities in complex, sequential autonomous systems, overcoming limitations of traditional Monte Carlo approaches.

Contribution

It presents a novel state-dependent importance sampling algorithm that minimizes divergence to improve failure probability estimation in large, sequential decision-making systems.

Findings

More accurate failure probability estimates than baseline methods

Effective on four different sequential systems

Open source implementation available

Abstract

Estimating the probability of failure is a critical step in developing safety-critical autonomous systems. Direct estimation methods such as Monte Carlo sampling are often impractical due to the rarity of failures in these systems. Existing importance sampling approaches do not scale to sequential decision-making systems with large state spaces and long horizons. We propose an adaptive importance sampling algorithm to address these limitations. Our method minimizes the forward Kullback-Leibler divergence between a state-dependent proposal distribution and a relaxed form of the optimal importance sampling distribution. Our method uses Markov score ascent methods to estimate this objective. We evaluate our approach on four sequential systems and show that it provides more accurate failure probability estimates than baseline Monte Carlo and importance sampling techniques. This work is open sourced.