Invariantly Admissible Policy Iteration for a Class of Nonlinear Optimal Control Problems

TL;DR

This paper introduces an invariantly admissible policy iteration method for nonlinear optimal control problems, ensuring admissibility and invariance without relying on common but often invalid assumptions, and demonstrating convergence and effectiveness through simulations.

Contribution

The paper proposes a novel invariantly admissible policy iteration method that guarantees admissibility and invariance without traditional assumptions, improving convergence in nonlinear control.

Findings

Guarantees admissibility and invariance of policies

Proves monotonic convergence of value functions

Demonstrates effectiveness through numerical simulations

Abstract

In this paper, we propose a generalized successive approximation method (SAM), called invariantly admissible policy iteration (PI), for finding the solution to a class of input-affine nonlinear optimal control problems by iterations. Unlike the existing SAM, the proposed method updates the domain of the next policy and value function for admissibility (and invariance). In the existing SAM, the admissibility of the generated policies are guaranteed under the two implicit assumptions regarding Lyapunov's theorem and invariance, both of which are presented and discussed in this paper and are generally not true. On the contrary, the proposed invariantly admissible PI guarantees the admissibility in a more refined manner, without such assumptions. The admissibility and invariance of the updated region, with respect to the corresponding policies, are mathematically prove under the specific invariant admissible update rule. We also provide monotonic decreasing and uniform convergence properties of the sequence of value functions under certain conditions. Finally, numerical simulations are presented to illustrate the proposed PI method and its effectiveness.