State and Input Constrained Output-Feedback Adaptive Optimal Control of Affine Nonlinear Systems

TL;DR

This paper introduces a novel online output-feedback reinforcement learning framework for safety-critical nonlinear systems that guarantees stability and safety without full-state measurements, using Lyapunov functions and barrier methods.

Contribution

It develops a critic-only, model-based RL approach with LMI-based observer design and safety-enforcing Lyapunov barrier functions, ensuring stability and safety in complex environments.

Findings

Successfully maintains system safety and stability in simulations.

Ensures state trajectories remain within safe sets.

Demonstrates obstacle avoidance in real-world scenarios.

Abstract

In this paper, a novel online, output-feedback, critic-only, model-based reinforcement learning framework is developed for safety-critical control systems operating in complex environments. The developed framework ensures system stability and safety, regardless of the lack of full-state measurement, while learning and implementing an optimal controller. The approach leverages linear matrix inequality-based observer design method to efficiently search for observer gains for effective state estimation. Then, approximate dynamic programming is used to develop an approximate controller that uses simulated experiences to guarantee the safety and stability of the closed-loop system. Safety is enforced by adding a recentered robust Lyapunov-like barrier function to the cost function that effectively enforces safety constraints, even in the presence of uncertainty in the state. Lyapunov-based stability analysis is used to guarantee uniform ultimate boundedness of the trajectories of the closed-loop system and ensure safety. Simulation studies are performed to demonstrate the effectiveness of the developed method through two real-world safety-critical scenarios, ensuring that the state trajectories of a given system remain in a given set and obstacle avoidance.