# Switched Linear Systems Meet Markov Decision Processes: Stability Guaranteed Policy Synthesis

## TL;DR

This paper addresses the synthesis of stabilizing policies for Markov decision process-driven switched linear systems, proposing three computational methods including bilinear matrix inequalities and convex optimization for improved scalability.

## Contribution

It introduces three novel approaches for policy synthesis in MDP-governed switched linear systems, extending stability conditions and enhancing computational efficiency.

## Key findings

- Proposed three methods for stabilizing policy synthesis.
- Extended stability conditions to MDP-driven systems.
- Compared approaches using three illustrative examples.

## Abstract

Switched linear systems are time-varying nonlinear systems whose dynamics switch between different modes, where each mode corresponds to different linear dynamics. They arise naturally to model unexpected failures, environment uncertainties or system noises during system operation. In this paper, we consider a special class of switched linear systems where the mode switches are governed by Markov decision processes (MDPs). We study the problem of synthesizing a policy in an MDP that stabilizes the switched system. Given a policy, the switched linear system becomes a Markov jump linear system whose stability conditions have been extensively studied. We make use of these stability conditions and propose three different computation approaches to find the stabilizing policy in an MDP. We derive our first approach by extending the stability conditions for a Markov jump linear system to handle switches governed by an MDP. This approach requires finding a feasible solution of a set of bilinear matrix inequalities, which makes policy synthesis typically challenging. To improve scalability, we provide two approaches based on convex optimization. We give three examples to show and compare our proposed solutions.