Mean-Field Controllability and Decentralized Stabilization of Markov Chains, Part I: Global Controllability and Rational Feedbacks

TL;DR

This paper investigates the controllability and stabilization of Markov chains on finite states, introducing decentralized feedback laws that respect graph structures, with potential applications in robotic swarm control.

Contribution

It establishes controllability results and develops decentralized rational feedback laws for stabilizing Markov chains, extending control design methods to structured networks.

Findings

Proves controllability from and to positive equilibria on strongly connected graphs.

Designs decentralized linear feedback laws for stabilization respecting graph constraints.

Demonstrates LMI-based construction of controllers for robotic swarm stabilization.

Abstract

In this paper, we study the controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters. Firstly, we prove small-time local and global controllability from and to strictly positive equilibrium configurations when the underlying graph is strongly connected. Secondly, we show that there always exists a locally exponentially stabilizing decentralized linear (density-)feedback law that takes zero valu at equilibrium and respects the graph structure, provided that the transition rates are allowed to be negative and the desired target density lies in the interior of the set of probability densities. For bidirected graphs, that is, graphs where a directed edge in one direction implies an edge in the opposite direction, we show that this linear control law can be realized using a decentralized rational feedback law of the form k(x) = a(x) + b(x)f(x)/g(x) that also respects the graph structure and control constraints (positivity and zero at equilibrium). This enables the possibility of using Linear Matrix Inequality (LMI) based tools to algorithmically construct decentralized density feedback controllers for stabilization of a robotic swarm to a target task distribution with no task-switching at equilibrium, as we demonstrate with several numerical examples.