Data-Driven Stochastic Control via Non-i.i.d. Trajectories: Foundations and Guarantees

TL;DR

This paper introduces a data-driven stochastic control framework using non-i.i.d. trajectories, providing probabilistic safety guarantees for systems with unknown dynamics and noise distributions, validated through multiple benchmarks.

Contribution

It develops a novel framework leveraging non-i.i.d. trajectory data and stochastic control barrier certificates, with SOS optimization for probabilistic safety guarantees, advancing beyond robust worst-case methods.

Findings

Validated on three stochastic benchmarks with unknown models.

Achieved probabilistic safety guarantees where robust methods fail.

Provided a safety controller with certified probabilistic satisfaction.

Abstract

This work establishes a crucial step toward advancing data-driven trajectory-based methods for stochastic systems with unknown mathematical dynamics. In contrast to scenario-based approaches that rely on independent and identically distributed (i.i.d.) trajectories, this work develops a data-driven framework where each trajectory is gathered over a finite horizon and exhibits temporal dependence-referred to as a non-i.i.d. trajectory. To ensure safety of dynamical systems using such trajectories, the current body of literature primarily considers dynamics subject to unknown-but-bounded disturbances, which facilitates robust analysis. While promising, such bounds may be violated in practice and the resulting worst-case robust analysis tends to be overly conservative. To overcome these fundamental challenges, this paper considers stochastic systems with unknown mathematical dynamics, influenced by process noise with unknown distributions. In the proposed framework, data is collected from stochastic systems under multiple realizations within a finite-horizon experiment, where each realization generates a non-i.i.d. trajectory. Leveraging the concept of stochastic control barrier certificates constructed from data, this work quantifies probabilistic safety guarantees with a certified confidence level. To achieve this, the proposed conditions are formulated as sum-of-squares (SOS) optimization problems, relying solely on empirical average of the collected trajectories and statistical features of the process noise. The efficacy of the approach has been validated on three stochastic benchmarks with both unknown models and noise distributions. In one case study, it is shown that while no safety controller exists for the robust analysis of the system under bounded disturbances, the proposed stochastic framework offers a safety controller with guaranteed probabilistic satisfaction.