SReachTools Kernel Module: Data-Driven Stochastic Reachability Using Hilbert Space Embeddings of Distributions

TL;DR

This paper introduces a data-driven approach for stochastic reachability analysis using kernel embeddings of distributions, integrated into SReachTools, enabling efficient safety probability computations for complex systems.

Contribution

It develops a novel kernel embedding-based method for stochastic reachability, extending SReachTools with algorithms that handle high-dimensional and black-box systems.

Findings

Effective in high-dimensional systems like a million-dimensional quadrotor

Provides finite sample bounds and convergence guarantees

Demonstrates applicability on diverse systems including cart-pole with neural controllers

Abstract

We present algorithms for performing data-driven stochastic reachability as an addition to SReachTools, an open-source stochastic reachability toolbox. Our method leverages a class of machine learning techniques known as kernel embeddings of distributions to approximate the safety probabilities for a wide variety of stochastic reachability problems. By representing the probability distributions of the system state as elements in a reproducing kernel Hilbert space, we can learn the "best fit" distribution via a simple regularized least-squares problem, and then compute the stochastic reachability safety probabilities as simple linear operations. This technique admits finite sample bounds and has known convergence in probability. We implement these methods as part of SReachTools, and demonstrate their use on a double integrator system, on a million-dimensional repeated planar quadrotor system, and a cart-pole system with a black-box neural network controller.