Model-free Nearly Optimal Control of Constrained-Input Nonlinear Systems Based on Synchronous Reinforcement Learning

TL;DR

This paper introduces a model-free reinforcement learning algorithm that learns nearly optimal control policies for constrained-input nonlinear systems directly from online data, ensuring stability and convergence without prior system knowledge.

Contribution

It presents a novel algorithm combining generalized policy iteration with neural networks, guaranteeing stability and convergence for constrained-input nonlinear systems.

Findings

Achieves nearly optimal control without system models

Ensures closed-loop stability via Lyapunov analysis

Converges reliably during online learning

Abstract

In this paper a novel model-free algorithm is proposed. This algorithm can learn the nearly optimal control law of constrained-input systems from online data without requiring any a priori knowledge of system dynamics. Based on the concept of generalized policy iteration method, there are two neural networks (NNs), namely actor and critic NN to approximate the optimal value function and optimal policy. The stability of closed-loop systems and the convergence of weights are also guaranteed by Lyapunov analysis.