Learning Optimal Feedback Operators and their Polynomial Approximation

TL;DR

This paper introduces a learning-based method to approximate optimal feedback laws for nonlinear control problems using polynomial ansatz, with convergence analysis and a graph-theoretic algorithm for efficiency, addressing the curse of dimensionality.

Contribution

It proposes a novel polynomial approximation approach with convergence guarantees and a graph-based evaluation method for efficient learning of feedback laws.

Findings

The method effectively approximates feedback laws for nonlinear control.

The polynomial ansatz converges under certain conditions.

The graph-theoretic algorithm improves computational efficiency.

Abstract

A learning based method for obtaining feedback laws for nonlinear optimal control problems is proposed. The learning problem is posed such that the open loop value function is its optimal solution. This infinite dimensional, function space, problem, is approximated by a polynomial ansatz and its convergence is analyzed. An $\ell_1$ penalty term is employed, which combined with the proximal point method, allows to find sparse solutions for the learning problem. The approach requires multiple evaluations of the elements of the polynomial basis and of their derivatives. In order to do this efficiently a graph-theoretic algorithm is devised. Several examples underline that the proposed methodology provides a promising approach for mitigating the curse of dimensionality which would be involved in case the optimal feedback law was obtained by solving the Hamilton Jacobi Bellman equation.