Reinforcement Learning-based Disturbance Rejection Control for Uncertain Nonlinear Systems

TL;DR

This paper presents a reinforcement learning-based control method for uncertain nonlinear systems that combines an extended state observer with online policy approximation, avoiding the need for persistence of excitation.

Contribution

It introduces a novel RL-based disturbance rejection control framework that estimates uncertainties in real time without requiring PE conditions, with proven convergence.

Findings

Effective disturbance rejection demonstrated in simulations

System state converges to the origin

Developed policy approaches the optimal policy

Abstract

This paper investigates the reinforcement learning (RL) based disturbance rejection control for uncertain nonlinear systems having non-simple nominal models. An extended state observer (ESO) is first designed to estimate the system state and the total uncertainty, which represents the perturbation to the nominal system dynamics. Based on the output of the observer, the control compensates for the total uncertainty in real time, and simultaneously, online approximates the optimal policy for the compensated system using a simulation of experience based RL technique. Rigorous theoretical analysis is given to show the practical convergence of the system state to the origin and the developed policy to the ideal optimal policy. It is worth mentioning that, the widely-used restrictive persistence of excitation (PE) condition is not required in the established framework. Simulation results are presented to illustrate the effectiveness of the proposed method.