Linear Quadratic Optimal Control and Stabilization for Discrete-time Markov Jump Linear Systems

TL;DR

This paper explores optimal control and stabilization of discrete-time Markov jump linear systems, providing explicit conditions and solutions for finite and infinite horizons using Riccati equations and stochastic difference equations.

Contribution

It offers necessary and sufficient conditions for optimal control and stabilization, including explicit solutions and the use of coupled Riccati equations for Markov jump systems.

Findings

Finite-horizon optimal controller characterized by coupled difference Riccati equations.

Infinite-horizon stabilization condition linked to the unique positive solution of algebraic Riccati equations.

Explicit expressions for optimal control and cost function in the infinite-horizon case.

Abstract

This paper mainly investigates the optimal control and stabilization problems for linear discrete-time Markov jump systems. The general case for the finite-horizon optimal controller is considered, where the input weighting matrix in the performance index is just required to be positive semi-definite. The necessary and sufficient condition for the existence of the optimal controller in finite-horizon is given explicitly from a set of coupled difference Riccati equations (CDRE). One of the key techniques is to solve the forward and backward stochastic difference equation (FDSDE) which is obtained by the maximum principle. As to the infinite-horizon case, we establish the necessary and sufficient condition to stabilize the Markov jump linear system in the mean square sense. It is shown that the Markov jump linear system is stabilizable under the optimal controller if and only if the associated couple algebraic Riccati equation (CARE) has a unique positive solution. Meanwhile, the optimal controller and optimal cost function in infinite-horizon case are expressed explicitly.