Absorption in Time-Varying Markov Chains: Graph-Based Conditions

TL;DR

This paper provides graph-based conditions to determine absorption and stabilizability in time-varying Markov chains, enabling verification of convergence to absorbing states under arbitrary or controlled switching.

Contribution

It introduces necessary and sufficient graph conditions for absorption and stabilizability in non-homogeneous Markov chains with switching policies.

Findings

Existence of switching policies ensuring absorption depends on reachability in union of transition graphs.

Three sufficient conditions for absorption under arbitrary switching are proposed.

Graph-theoretic conditions can verify stability based solely on transition feasibility.

Abstract

We investigate absorption, i.e., almost sure convergence to an absorbing state, in time-varying (non-homogeneous) discrete-time Markov chains with finite state space. We consider systems that can switch among a finite set of transition matrices, which we call the modes. Our analysis is focused on two properties: 1) almost sure convergence to an absorbing state under any switching, and 2) almost sure convergence to a desired set of absorbing states via a proper switching policy. We derive necessary and sufficient conditions based on the structures of the transition graphs of modes. More specifically, we show that a switching policy that ensures almost sure convergence to a desired set of absorbing states from any initial state exists if and only if those absorbing states are reachable from any state on the union of simplified transition graphs. We then show three sufficient conditions for absorption under arbitrary switching. While the first two conditions depend on the acyclicity (weak acyclicity) of the union (intersection) of simplified transition graphs, the third condition is based on the distances of each state to the absorbing states in all the modes. These graph theoretic conditions can verify the stability and stabilizability of absorbing states based only on the feasibility of transitions in each mode.