Stability Analysis of Optimal Adaptive Control using Value Iteration with Approximation Errors

TL;DR

This paper provides a theoretical stability analysis of adaptive optimal control using value iteration, accounting for approximation errors and offering estimates of the region of attraction to ensure system stability during learning.

Contribution

It introduces a stability analysis framework for adaptive control with approximation errors, applicable to both fixed and evolving control policies, and estimates the region of attraction.

Findings

Stability during learning is maintained within the estimated region of attraction.

The analysis accounts for approximation errors in value iteration-based control.

Applicable to both constant and time-varying control policies.

Abstract

Adaptive optimal control using value iteration initiated from a stabilizing control policy is theoretically analyzed in terms of stability of the system during the learning stage without ignoring the effects of approximation errors. This analysis includes the system operated using any single/constant resulting control policy and also using an evolving/time-varying control policy. A feature of the presented results is providing estimations of the \textit{region of attraction} so that if the initial condition is within the region, the whole trajectory will remain inside it and hence, the function approximation results remain valid.