Weighted Stochastic Riccati Equations for Generalization of Linear Optimal Control

TL;DR

This paper introduces weighted stochastic Riccati equations to design diverse optimal controllers for linear stochastic systems with i.i.d. matrices, enabling robust and risk-sensitive control policies despite system uncertainties.

Contribution

It proposes a novel WSR framework that generalizes Riccati equations for multiple control types and introduces efficient solution methods and a new robust RSL controller.

Findings

WSR equations unify deterministic, stochastic, and risk-sensitive control.

Efficient iterative and Newton's method solutions are developed.

The robust RSL controller offers combined risk sensitivity and robustness.

Abstract

This paper presents weighted stochastic Riccati (WSR) equations for designing multiple types of optimal controllers for linear stochastic systems. The stochastic system matrices are independent and identically distributed (i.i.d.) to represent uncertainty and noise in the systems. However, it is difficult to design multiple types of controllers for systems with i.i.d. matrices while the stochasticity can invoke unpredictable control results. A critical limitation of such i.i.d. systems is that Riccati-like algebraic equations cannot be applied to complex controller design. To overcome this limitation, the proposed WSR equations employ a weighted expectation of stochastic algebraic equations. The weighted expectation is calculated using a weight function designed to handle statistical properties of the control policy. Solutions to the WSR equations provide multiple policies depending on the weight function, which contain the deterministic optimal, stochastic optimal, and risk-sensitive linear (RSL) control. This study presents two approaches to solve the WSR equations efficiently: calculating WSR difference equations iteratively and employing Newton's method. Moreover, designing the weight function yields a novel controller termed the robust RSL controller that has both a risk-sensitive policy and robustness to randomness occurring in stochastic control design.