Data-Driven Optimal Control via Linear Transfer Operators: A Convex Approach

TL;DR

This paper introduces a convex, data-driven method for optimal control of nonlinear systems by lifting dynamics into a density space using linear transfer operators, enabling efficient polynomial optimization.

Contribution

It develops a novel convex formulation of the nonlinear optimal control problem using the Perron-Frobenius and Koopman operators, with a practical approximation via sum-of-squares optimization.

Findings

Effective control framework demonstrated through simulations.

Convex formulation handles both positive and negative discount factors.

Polynomial optimization approach enables efficient computation.

Abstract

This paper is concerned with data-driven optimal control of nonlinear systems. We present a convex formulation to the optimal control problem (OCP) with a discounted cost function. We consider OCP with both positive and negative discount factor. The convex approach relies on lifting nonlinear system dynamics in the space of densities using the linear Perron-Frobenius (P-F) operator. This lifting leads to an infinite-dimensional convex optimization formulation of the optimal control problem. The data-driven approximation of the optimization problem relies on the approximation of the Koopman operator using the polynomial basis function. We write the approximate finite-dimensional optimization problem as a polynomial optimization which is then solved efficiently using a sum-of-squares-based optimization framework. Simulation results are presented to demonstrate the efficacy of the developed data-driven optimal control framework.