Reinforcement Learning-Based Optimal Control for Multiplicative-Noise Systems with Input Delay

TL;DR

This paper introduces a reinforcement learning approach to design optimal controllers for systems with multiplicative noise and input delays, even when some system dynamics are unknown, by solving a modified Riccati equation.

Contribution

It develops a stabilizing condition and a convergent RL-based algorithm for solving Riccati-ZXL equations in systems with input delay and partial dynamics knowledge.

Findings

The proposed algorithm converges reliably in numerical tests.

It effectively handles systems with multiplicative noise and input delay.

The method enables optimal control without full system knowledge.

Abstract

In this paper, the reinforcement learning (RL)-based optimal control problem is studied for multiplicative-noise systems, where input delay is involved and partial system dynamics is unknown. To solve a variant of Riccati-ZXL equations, which is a counterpart of standard Riccati equation and determines the optimal controller, we first develop a necessary and sufficient stabilizing condition in form of several Lyapunov-type equations, a parallelism of the classical Lyapunov theory. Based on the condition, we provide an offline and convergent algorithm for the variant of Riccati-ZXL equations. According to the convergent algorithm, we propose a RL-based optimal control design approach for solving linear quadratic regulation problem with partially unknown system dynamics. Finally, a numerical example is used to evaluate the proposed algorithm.