Output Feedback Adaptive Optimal Control of Affine Nonlinear systems with a Linear Measurement Model

TL;DR

This paper introduces an online output-feedback reinforcement learning method for affine nonlinear systems that learns optimal control policies while ensuring stability and robustness in uncertain environments.

Contribution

It proposes a critic-only, model-based reinforcement learning architecture with a new observer gain search method, ensuring stability and convergence during learning.

Findings

Ensures stability and convergence of the control system during learning.

Demonstrates local boundedness of trajectories through Lyapunov analysis.

Validates approach with simulation results under mild excitation conditions.

Abstract

Real-world control applications in complex and uncertain environments require adaptability to handle model uncertainties and robustness against disturbances. This paper presents an online, output-feedback, critic-only, model-based reinforcement learning architecture that simultaneously learns and implements an optimal controller while maintaining stability during the learning phase. Using multiplier matrices, a convenient way to search for observer gains is designed along with a controller that learns from simulated experience to ensure stability and convergence of trajectories of the closed-loop system to a neighborhood of the origin. Local uniform ultimate boundedness of the trajectories is established using a Lyapunov-based analysis and demonstrated through simulation results, under mild excitation conditions.