Optimal feedback control of dynamical systems via value-function approximation

TL;DR

This paper introduces a self-learning method for designing optimal feedback controllers for nonlinear systems using neural network approximations of the value function, leveraging the connection between maximum principle and dynamic programming.

Contribution

It presents a novel approach combining universal approximation, ensemble averaging, and optimal control theory to learn feedback laws with proven convergence and optimality conditions.

Findings

Proves existence and convergence of neural network feedback controllers.

Establishes first-order optimality conditions for the proposed method.

Demonstrates the effectiveness of the approach through theoretical analysis.

Abstract

A self-learning approach for optimal feedback gains for finite-horizon nonlinear continuous time control systems is proposed and analysed. It relies on parameter dependent approximations to the optimal value function obtained from a family of universal approximators. The cost functional for the training of an approximate optimal feedback law incorporates two main features. First, it contains the average over the objective functional values of the parametrized feedback control for an ensemble of initial values. Second, it is adapted to exploit the relationship between the maximum principle and dynamic programming. Based on universal approximation properties, existence, convergence and first order optimality conditions for optimal neural network feedback controllers are proved.