Neighboring Extremal Optimal Control Theory for Parameter-Dependent Closed-loop Laws

TL;DR

This paper presents a novel method for efficiently computing neighboring extremal optimal controls in nonlinear systems with parameter variations, using linear PDE solutions and a Galerkin-based numerical algorithm.

Contribution

It introduces a linear PDE approach for NEOC, a Galerkin numerical algorithm, and a homotopic method for large parameter changes, simplifying the computation process.

Findings

The method accurately computes control law adjustments for small parameter variations.

The Galerkin algorithm effectively solves the Hamilton-Jacobi equation.

The homotopic approach handles large parameter perturbations successfully.

Abstract

This study introduces an approach to obtain a neighboring extremal optimal control (NEOC) solution for a closed-loop optimal control problem, applicable to a wide array of nonlinear systems and not necessarily quadratic performance indices. The approach involves investigating the variation incurred in the functional form of a known closed-loop optimal control law due to small, known parameter variations in the system equations or the performance index. The NEOC solution can formally be obtained by solving a linear partial differential equation, akin to those encountered in the iterative solution of a nonlinear Hamilton-Jacobi equation. Motivated by numerical procedures for solving these latter equations, we also propose a numerical algorithm based on the Galerkin algorithm, leveraging the use of basis functions to solve the underlying Hamilton-Jacobi equation of the original optimal control problem. The proposed approach simplifies the NEOC problem by reducing it to the solution of a simple set of linear equations, thereby eliminating the need for a full re-solution of the adjusted optimal control problem. Furthermore, the variation to the optimal performance index can be obtained as a function of both the system state and small changes in parameters, allowing the determination of the adjustment to an optimal control law given a small adjustment of parameters in the system or the performance index. Moreover, in order to handle large known parameter perturbations, we propose a homotopic approach that breaks down the single calculation of NEOC into a finite set of multiple steps. Finally, the validity of the claims and theory is supported by theoretical analysis and numerical simulations.