Deep Reinforcement Learning Optimization for Uncertain Nonlinear Systems via Event-Triggered Robust Adaptive Dynamic Programming

TL;DR

This paper introduces a control framework combining reinforcement learning, disturbance estimation, and event-triggered updates to efficiently manage uncertain nonlinear systems with reduced computational effort.

Contribution

It presents a novel integrated control architecture that uses event-triggered mechanisms with adaptive dynamic programming and disturbance observers for improved efficiency and robustness.

Findings

Maintains strong control performance under disturbances.

Reduces computational load significantly compared to traditional methods.

Ensures system stability through Lyapunov analysis.

Abstract

This work proposes a unified control architecture that couples a Reinforcement Learning (RL)-driven controller with a disturbance-rejection Extended State Observer (ESO), complemented by an Event-Triggered Mechanism (ETM) to limit unnecessary computations. The ESO is utilized to estimate the system states and the lumped disturbance in real time, forming the foundation for effective disturbance compensation. To obtain near-optimal behavior without an accurate system description, a value-iteration-based Adaptive Dynamic Programming (ADP) method is adopted for policy approximation. The inclusion of the ETM ensures that parameter updates of the learning module are executed only when the state deviation surpasses a predefined bound, thereby preventing excessive learning activity and substantially reducing computational load. A Lyapunov-oriented analysis is used to characterize the stability properties of the resulting closed-loop system. Numerical experiments further confirm that the developed approach maintains strong control performance and disturbance tolerance, while achieving a significant reduction in sampling and processing effort compared with standard time-triggered ADP schemes.