A Short Proof of a Convex Representation for Stationary Distributions of Markov Chains with an Application to State Space Truncation

TL;DR

This paper provides a concise proof of a convex representation for stationary distributions of Markov chains, extending the theory to infinite and continuous state spaces, with applications to error bounds in state truncation algorithms.

Contribution

It offers a simplified proof of a key convex representation, extending it to broader state space settings and applying it to derive new error bounds for Markov chain truncations.

Findings

Extended the convex representation to infinite and continuous state spaces.

Derived a new total variation distance error bound for truncated Markov chains.

Enhanced understanding of error analysis in Markov chain algorithms.

Abstract

In an influential paper, Courtois and Semal (1984) establish that when $G$ is an irreducible substochastic matrix for which $\sum_{n=0}^{\infty}G^n <\infty$, then the stationary distribution of any stochastic matrix $P\ge G$ can be expressed as a convex combination of the normalized rows of $(I-G)^{-1} = \sum_{n=0}^{\infty} G^n$. In this note, we give a short proof of this result that extends the theory to the countably infinite and continuous state space settings. This result plays an important role in obtaining error bounds in algorithms involving nearly decomposable Markov chains, and also in state truncations for Markov chains. We also use the representation to establish a new total variation distance error bound for truncated Markov chains.