An Online Model-Following Projection Mechanism Using Reinforcement Learning

TL;DR

This paper introduces a real-time, model-free adaptive control method using reinforcement learning and a projection mechanism to improve model-following performance in nonlinear systems.

Contribution

It presents a novel online reinforcement learning approach with a projection mechanism for adaptive control, enhancing real-time tuning and performance in model-following tasks.

Findings

Outperforms sliding mode control in simulations

Provides stable real-time adaptation in nonlinear systems

Demonstrates improved convergence of control strategies

Abstract

In this paper, we propose a model-free adaptive learning solution for a model-following control problem. This approach employs policy iteration, to find an optimal adaptive control solution. It utilizes a moving finite-horizon of model-following error measurements. In addition, the control strategy is designed by using a projection mechanism that employs Lagrange dynamics. It allows for real-time tuning of derived actor-critic structures to find the optimal model-following strategy and sustain optimized adaptation performance. Finally, the efficacy of the proposed framework is emphasized through a comparison with sliding mode and high-order model-free adaptive control approaches. Keywords: Model Reference Adaptive Systems, Reinforcement Learning, adaptive critics, control system, stochastic, nonlinear system