Multi-scale metastable dynamics and the asymptotic stationary distribution of perturbed Markov chains

TL;DR

This paper analyzes perturbed Markov chains exhibiting metastable behavior across multiple time scales, providing explicit formulas and efficient algorithms for their asymptotic transition probabilities and stationary distributions.

Contribution

It introduces a framework for deriving effective chains that describe metastable dynamics and escape probabilities in perturbed Markov chains, with practical computational methods.

Findings

Explicit formulas for asymptotic transition probabilities.

Efficient algorithms for computing the stationary distribution.

Analysis of metastable behavior on multiple time scales.

Abstract

We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$. We further assume that the chain is irreducible for $\varepsilon$ but may have several essential communicating classes when $\varepsilon$. This leads to metastable behavior, possibly on multiple time scales. For each of the relevant time scales, we derive two effective chains. The first one describes the (possibly irreversible) metastable dynamics, while the second one is reversible and describes metastable escape probabilities. Closed probabilistic expressions are given for the asymptotic transition probabilities of these chains, but we also show how to compute them in a fast and numerically stable way. As a consequence, we obtain efficient algorithms for computing the committor function and the limiting stationary distribution.