Stationary distributions of continuous-time Markov chains: a review of theory and truncation-based approximations

TL;DR

This paper reviews the theory of stationary distributions of continuous-time Markov chains and examines truncation-based approximation methods, highlighting their convergence, errors, and practical applications in biological and chemical systems.

Contribution

It provides a comprehensive, accessible review of the theory and practical approximation schemes for CTMC stationary distributions, including convergence and error analysis.

Findings

Truncation schemes can effectively approximate stationary distributions.

Error bounds depend on the truncation method and system properties.

Application to biological and chemical reaction networks demonstrates practical utility.

Abstract

Computing the stationary distributions of a continuous-time Markov chain (CTMC) involves solving a set of linear equations. In most cases of interest, the number of equations is infinite or too large, and the equations cannot be solved analytically or numerically. Several approximation schemes overcome this issue by truncating the state space to a manageable size. In this review, we first give a comprehensive theoretical account of the stationary distributions and their relation to the long-term behaviour of CTMCs that is readily accessible to non-experts and free of irreducibility assumptions made in standard texts. We then review truncation-based approximation schemes for CTMCs with infinite state spaces paying particular attention to the schemes' convergence and the errors they introduce, and we illustrate their performance with an example of a stochastic reaction network of relevance in biology and chemistry. We conclude by discussing computational trade-offs associated with error control and several open questions.