Finite-Horizon LQR for General Markov Jump Linear Systems: Deterministic Reformulation and Reduced-Order Solution

TL;DR

This paper develops a deterministic reformulation and reduced-order solution for the finite-horizon LQR problem in Markov Jump Linear Systems with complex state structures, enabling efficient control design.

Contribution

It introduces a novel deterministic approach using second-moment matrices and a projection operator to handle general Markov chains, including non-communicating states, for finite-horizon LQR.

Findings

Reduced-order Riccati equations efficiently solve the control problem.

The approach handles non-communicating and transient states.

Numerical examples validate the theoretical framework.

Abstract

This paper studies the Linear Quadratic Regulator (LQR) problem for continuous-time Markov Jump Linear Systems (MJLS) governed by general finite-state Markov chains that may include transient, absorbing, or non-communicating states. The problem, posed over a finite time horizon, is reformulated deterministically by expressing the cost functional in terms of a collection of second-moment matrices of the system state. A projection operator is introduced to restrict the analysis to the subspace of visited states, namely those with positive probability of being reached within the time horizon. The solution of the resulting deterministic problem is obtained from a reduced-order system of coupled matrix Riccati differential equations involving only the visited states, which define a quadratic value function satisfying a Hamilton-Jacobi-Bellman type equation. The structure of this equation is formally justified in the matrix setting via the Riesz-Frechet representation theorem, establishing a rigorous foundation for the deterministic reformulation and resolving an open aspect in previous literature. Several numerical examples, including cases with non-communicating states, validate the theoretical results and illustrate the practical relevance of the proposed generalization.