Stochastic Optimal Linear Quadratic Regulation Control of Discrete-time Systems with Delay and Quadratic Constraints

TL;DR

This paper develops a method for optimal control of discrete-time stochastic systems with delays and quadratic constraints, using Riccati-ZXL equations and gradient ascent to derive feedback controllers.

Contribution

It introduces a novel approach combining Riccati-ZXL equations and gradient ascent for delay-inclusive stochastic LQR control with quadratic constraints.

Findings

Effective feedback control derived from Riccati-ZXL equations.

Gradient ascent successfully determines optimal controller parameters.

Numerical examples validate the proposed method's effectiveness.

Abstract

This article explores the discrete-time stochastic optimal LQR control with delay and quadratic constraints. The inclusion of delay, compared to delay-free optimal LQR control with quadratic constraints, significantly increases the complexity of the problem. Using Lagrangian duality, the optimal control is obtained by solving the Riccati-ZXL equation in conjunction with a gradient ascent algorithm. Specifically, the parameterized optimal controller and cost function are derived by solving the Riccati-ZXL equation, with a gradient ascent algorithm determining the optimal parameter. The primary contribution of this work is presenting the optimal control as a feedback mechanism based on the state's conditional expectation, wherein the gain is determined using the Riccati-ZXL equation and the gradient ascent algorithm. Numerical examples demonstrate the effectiveness of the obtained results.