Ensemble based Closed-Loop Optimal Control using Physics-Informed Neural Networks

TL;DR

This paper introduces an ensemble physics-informed neural network framework for solving the Hamilton-Jacobi-Bellman equation to design optimal control systems, enabling effective closed-loop control of nonlinear systems without stabilizer terms.

Contribution

It proposes a multistage ensemble PINN approach for learning optimal control policies directly from the HJB equation, avoiding stabilizer terms used in prior methods.

Findings

Successfully controls a nonlinear system with noisy states.

Demonstrates ensemble and singular control effectiveness.

Handles varying initial conditions and perturbations.

Abstract

The objective of designing a control system is to steer a dynamical system with a control signal, guiding it to exhibit the desired behavior. The Hamilton-Jacobi-Bellman (HJB) partial differential equation offers a framework for optimal control system design. However, numerical solutions to this equation are computationally intensive, and analytical solutions are frequently unavailable. Knowledge-guided machine learning methodologies, such as physics-informed neural networks (PINNs), offer new alternative approaches that can alleviate the difficulties of solving the HJB equation numerically. This work presents a multistage ensemble framework to learn the optimal cost-to-go, and subsequently the corresponding optimal control signal, through the HJB equation. Prior PINN-based approaches rely on a stabilizing the HJB enforcement during training. Our framework does not use stabilizer terms and offers a means of controlling the nonlinear system, via either a singular learned control signal or an ensemble control signal policy. Success is demonstrated in closed-loop control, using both ensemble- and singular-control, of a steady-state time-invariant two-state continuous nonlinear system with an infinite time horizon, accounting of noisy, perturbed system states and varying initial conditions.