Data-Driven Control via Conditional Mean Embeddings: Formal Guarantees via Uncertain MDP Abstraction

TL;DR

This paper introduces a data-driven control framework that uses conditional mean embeddings and uncertain MDPs to provide formal performance guarantees for stochastic systems with unknown dynamics, validated on a temperature regulation benchmark.

Contribution

It develops a novel approach combining CMEs and UMDPs for policy synthesis with formal guarantees in uncertain, data-driven control settings.

Findings

Successfully learns transition kernels from data

Constructs finite-state UMDP abstractions with uncertainty bounds

Provides formal performance guarantees through robust dynamic programming

Abstract

Controlling stochastic systems with unknown dynamics and under complex specifications is specially challenging in safety-critical settings, where performance guarantees are essential. We propose a data-driven policy synthesis framework that yields formal performance guarantees for such systems using conditional mean embeddings (CMEs) and uncertain Markov decision processes (UMDPs). From trajectory data, we learn the system's transition kernel as a CME, then construct a finite-state UMDP abstraction whose transition uncertainties capture learning and discretization errors. Next, we generate a policy with formal performance bounds through robust dynamic programming. We demonstrate and empirically validate our method through a temperature regulation benchmark.