Statistically Assuring Safety of Control Systems using Ensembles of Safety Filters and Conformal Prediction

TL;DR

This paper introduces a conformal prediction framework to provide probabilistic safety guarantees for control systems using learned Hamilton-Jacobi value functions, and explores ensemble methods to improve safety assurance in autonomous systems.

Contribution

It proposes a novel conformal prediction-based approach to bound uncertainty in learned safety functions and compares ensemble versus individual safety filters for control.

Findings

CP provides probabilistic safety guarantees for learned policies.

Ensemble safety filters outperform individual ones in safety assurance.

The framework enables safer deployment of learning-enabled autonomous systems.

Abstract

Safety assurance is a fundamental requirement for deploying learning-enabled autonomous systems. Hamilton-Jacobi (HJ) reachability analysis is a fundamental method for formally verifying safety and generating safe controllers. However, computing the HJ value function that characterizes the backward reachable set (BRS) of a set of user-defined failure states is computationally expensive, especially for high-dimensional systems, motivating the use of reinforcement learning approaches to approximate the value function. Unfortunately, a learned value function and its corresponding safe policy are not guaranteed to be correct. The learned value function evaluated at a given state may not be equal to the actual safety return achieved by following the learned safe policy. To address this challenge, we introduce a conformal prediction-based (CP) framework that bounds such uncertainty. We leverage CP to provide probabilistic safety guarantees when using learned HJ value functions and policies to prevent control systems from reaching failure states. Specifically, we use CP to calibrate the switching between the unsafe nominal controller and the learned HJ-based safe policy and to derive safety guarantees under this switched policy. We also investigate using an ensemble of independently trained HJ value functions as a safety filter and compare this ensemble approach to using individual value functions alone.