Adaptive Actor-Critic Based Optimal Regulation for Drift-Free Uncertain Nonlinear Systems

TL;DR

This paper introduces a continuous-time adaptive actor-critic reinforcement learning controller for drift-free nonlinear systems with parametric uncertainties, ensuring stability and near-optimal control in applications like visual servoing and mobile robots.

Contribution

It develops a novel RL controller with concurrent learning for unknown parameters, addressing input uncertainty challenges in drift-free nonlinear systems.

Findings

Guarantees closed-loop stability.

Validates effectiveness with IBVS and WMR simulations.

Achieves near-optimal control efforts.

Abstract

In this paper, a continuous-time adaptive actor-critic reinforcement learning (RL) controller is developed for drift-free nonlinear systems. Practical examples of such systems are image-based visual servoing (IBVS) and wheeled mobile robots (WMR), where the system dynamics includes a parametric uncertainty in the control effectiveness matrix with no drift term. The uncertainty in the input term poses a challenge for developing a continuous-time RL controller using existing methods. In this paper, an actor-critic or synchronous policy iteration (PI)-based RL controller is presented with a concurrent learning (CL)-based parameter update law for estimating the unknown parameters of the control effectiveness matrix. An infinite-horizon value function minimization objective is achieved by regulating the current states to the desired with near-optimal control efforts. The proposed controller guarantees closed-loop stability and simulation results validate the proposed theory using IBVS and WMR examples.