State-Based Confidence Bounds for Data-Driven Stochastic Reachability Using Hilbert Space Embeddings

TL;DR

This paper introduces a nonparametric, data-driven method using kernel embeddings to compute probabilistic safety bounds for stochastic systems, adaptable to non-uniform data sampling.

Contribution

It develops finite sample confidence bounds for stochastic reachability using Hilbert space embeddings, enabling model-free safety analysis with data-dependent tightness.

Findings

Effective in providing probabilistic safety guarantees

Handles non-uniformly sampled data for tighter bounds

Demonstrated on neural network-controlled pendulum

Abstract

In this paper, we compute finite sample bounds for data-driven approximations of the solution to stochastic reachability problems. Our approach uses a nonparametric technique known as kernel distribution embeddings, and provides probabilistic assurances of safety for stochastic systems in a model-free manner. By implicitly embedding the stochastic kernel of a Markov control process in a reproducing kernel Hilbert space, we can approximate the safety probabilities for stochastic systems with arbitrary stochastic disturbances as simple matrix operations and inner products. We present finite sample bounds for point-based approximations of the safety probabilities through construction of probabilistic confidence bounds that are state- and input-dependent. One advantage of this approach is that the bounds are responsive to non-uniformly sampled data, meaning that tighter bounds are feasible in regions of the state- and input-space with more observations. We numerically evaluate the approach, and demonstrate its efficacy on a neural network-controlled pendulum system.