Computing stabilizing feedback gains for stochastic linear systems via policy iteration method

TL;DR

This paper introduces a model-free reinforcement learning algorithm using policy iteration to compute stabilizing feedback gains for stochastic LTI systems with unknown parameters, ensuring stabilization after finite steps.

Contribution

It develops a novel policy iteration-based method that guarantees stabilization of stochastic LTI systems with unknown dynamics, advancing control in uncertain environments.

Findings

Algorithm successfully stabilizes systems in finite steps

Numerical example demonstrates effectiveness

Method applicable to unknown stochastic systems

Abstract

In recent years, stabilizing unknown dynamical systems has became a critical problem in control systems engineering. Addressing this for linear time-invariant (LTI) systems is an essential fist step towards solving similar problems for more complex systems. In this paper, we develop a model-free reinforcement learning algorithm to compute stabilizing feedback gains for stochastic LTI systems with unknown system matrices. This algorithm proceeds by solving a series of discounted stochastic linear quadratic (SLQ) optimal control problems via policy iteration (PI). And the corresponding discount factor gradually decreases according to an explicit rule, which is derived from the equivalent condition in verifying the stabilizability. We prove that this method can return a stabilizer after finitely many steps. Finally, a numerical example is provided to illustrate the effectiveness of the proposed method.