Neural Time-Reversed Generalized Riccati Equation

TL;DR

This paper presents a neural network-based method for optimal control that estimates costates forward-in-time using a novel local policy inspired by Riccati equations, enabling more efficient solutions.

Contribution

It introduces a new neural approach employing a time-reversed Riccati-inspired policy for estimating costates in optimal control, working forward-in-time.

Findings

Supports conjecture on stabilization of state dynamics

Demonstrates effectiveness across multiple case studies

Offers a forward-in-time alternative to traditional methods

Abstract

Optimal control deals with optimization problems in which variables steer a dynamical system, and its outcome contributes to the objective function. Two classical approaches to solving these problems are Dynamic Programming and the Pontryagin Maximum Principle. In both approaches, Hamiltonian equations offer an interpretation of optimality through auxiliary variables known as costates. However, Hamiltonian equations are rarely used due to their reliance on forward-backward algorithms across the entire temporal domain. This paper introduces a novel neural-based approach to optimal control, with the aim of working forward-in-time. Neural networks are employed not only for implementing state dynamics but also for estimating costate variables. The parameters of the latter network are determined at each time step using a newly introduced local policy referred to as the time-reversed generalized Riccati equation. This policy is inspired by a result discussed in the Linear Quadratic (LQ) problem, which we conjecture stabilizes state dynamics. We support this conjecture by discussing experimental results from a range of optimal control case studies.