Adaptive Splitting of Reusable Temporal Monitors for Rare Traffic Violations

TL;DR

This paper introduces an adaptive splitting method combining rare-event sampling and online monitoring to efficiently estimate AV safety violations, outperforming traditional Monte-Carlo and importance sampling techniques.

Contribution

It presents a novel adaptive multi-level splitting approach that leverages robustness metrics and caching for efficient rare-event probability estimation in AV simulations.

Findings

Outperforms Monte-Carlo in efficiency

Provides more accurate failure probability estimates

Applicable to real traffic rule scenarios

Abstract

Autonomous Vehicles (AVs) are often tested in simulation to estimate the probability they will violate safety specifications. Two common issues arise when using existing techniques to produce this estimation: If violations occur rarely, simple Monte-Carlo sampling techniques can fail to produce efficient estimates; if simulation horizons are too long, importance sampling techniques (which learn proposal distributions from past simulations) can fail to converge. This paper addresses both issues by interleaving rare-event sampling techniques with online specification monitoring algorithms. We use adaptive multi-level splitting to decompose simulations into partial trajectories, then calculate the distance of those partial trajectories to failure by leveraging robustness metrics from Signal Temporal Logic (STL). By caching those partial robustness metric values, we can efficiently re-use computations across multiple sampling stages. Our experiments on an interstate lane-change scenario show our method is viable for testing simulated AV-pipelines, efficiently estimating failure probabilities for STL specifications based on real traffic rules. We produce better estimates than Monte-Carlo and importance sampling in fewer simulations.