# Maximal Invariant Set Computation and Design for Markov Chains

## TL;DR

This paper introduces an algorithm for computing the maximal invariant set of Markov chains with safety constraints, optimizing convergence to steady-state distributions via SDP, and demonstrates its application in decentralized swarm guidance.

## Contribution

It presents a novel SDP-based method for efficient maximal invariant set computation and Markov chain synthesis with safety guarantees.

## Key findings

- Efficient SDP formulation for invariant set computation.
- Guaranteed finite determination of the invariant set.
- Successful application to decentralized swarm guidance.

## Abstract

We describe an algorithm for computing the maximal invariant set for a Markov chain with linear safety constraints on the distribution over states. We then propose a Markov chain synthesis method that guarantees finite determination of the maximal invariant set. Although this problem is bilinear in the general case, we are able to optimize the convergence rate to a desirable steady-state distribution over reversible Markov chains by solving a Semidefinite Program (SDP), which promotes efficient computation of the maximal invariant set. We then demonstrate this approach with a decentralized swarm guidance application subject to density upper bounds.