Discrete fully probabilistic design: towards a control pipeline for the synthesis of policies from examples

TL;DR

This paper introduces a discrete fully probabilistic design pipeline for synthesizing control policies from example data, capable of handling noisy, constrained, and cross-system data, demonstrated on an inverted pendulum example.

Contribution

It presents a novel control pipeline that does not require constraint satisfaction in data and can synthesize policies from data of different systems, expanding applicability.

Findings

Successfully controls an inverted pendulum from cross-system data.

Handles noisy and constrained data without explicit constraint satisfaction.

Openly shares the implementation code for reproducibility.

Abstract

We present the principled design of a control pipeline for the synthesis of policies from examples data. The pipeline, based on a discretized design which we term as discrete fully probabilistic design, expounds an algorithm recently introduced in Gagliardi and Russo (2021) to synthesize policies from examples for constrained, stochastic and nonlinear systems. Contrary to other approaches, the pipeline we present: (i) does not need the constraints to be fulfilled in the possibly noisy example data; (ii) enables control synthesis even when the data are collected from an example system that is different from the one under control. The design is benchmarked numerically on an example that involves controlling an inverted pendulum with actuation constraints starting from data collected from a physically different pendulum that does not satisfy the system-specific actuation constraints. We also make our fully documented code openly available.