Data-Driven Optimal Control of Affine Systems: A Linear Programming Perspective

TL;DR

This paper presents a data-driven linear programming approach for optimal control of affine systems, introducing a unified framework and methods to synthesize Bellman inequalities from limited data, extending Willem's fundamental lemma.

Contribution

It develops a unified fixed point framework for value functions and extends Willem's lemma to affine systems for model-free control synthesis.

Findings

Effective synthesis of Bellman inequalities from small datasets

Extension of Willem's fundamental lemma to affine systems

Unified framework for fixed point characterization

Abstract

In this letter, we discuss the problem of optimal control for affine systems in the context of data-driven linear programming. First, we introduce a unified framework for the fixed point characterization of the value function, Q-function and relaxed Bellman operators. Then, in a model-free setting, we show how to synthesize and estimate Bellman inequalities from a small but sufficiently rich dataset. To guarantee exploration richness, we complete the extension of Willem's fundamental lemma to affine systems.