A Physics-Informed Learning Framework to Solve the Infinite-Horizon Optimal Control Problem

TL;DR

This paper introduces a physics-informed neural network framework to solve the infinite-horizon optimal control problem by approximating the value function through a finite-horizon HJB equation, addressing solution uniqueness and computational efficiency.

Contribution

The authors develop a novel PINNs-based approach applying to a finite-horizon HJB to approximate the infinite-horizon value function, avoiding iterative policy evaluation and requiring no prior stabilizing controller.

Findings

Method effectively approximates the optimal value function.

Works well with non-polynomial basis functions.

Includes an algorithm to verify horizon sufficiency.

Abstract

We propose a physics-informed neural networks (PINNs) framework to solve the infinite-horizon optimal control problem of nonlinear systems. In particular, since PINNs are generally able to solve a class of partial differential equations (PDEs), they can be employed to learn the value function of the infinite-horizon optimal control problem via solving the associated steady-state Hamilton-Jacobi-Bellman (HJB) equation. However, an issue here is that the steady-state HJB equation generally yields multiple solutions; hence if PINNs are directly employed to it, they may end up approximating a solution that is different from the optimal value function of the problem. We tackle this by instead applying PINNs to a finite-horizon variant of the steady-state HJB that has a unique solution, and which uniformly approximates the optimal value function as the horizon increases. An algorithm to verify if the chosen horizon is large enough is also given, as well as a method to extend it -- with reduced computations and robustness to approximation errors -- in case it is not. Unlike many existing methods, the proposed technique works well with non-polynomial basis functions, does not require prior knowledge of a stabilizing controller, and does not perform iterative policy evaluations. Simulations are performed, which verify and clarify theoretical findings.