Perturbation bounds for the stationary distributions of Markov chains

TL;DR

This paper develops new perturbation bounds for stationary distributions of Markov chains, applicable to both discrete and continuous-time cases, using norm-wise bounds, drift functions, and mean hitting times, with practical examples.

Contribution

Introduces novel norm-wise perturbation bounds for Markov chains, including bounds for periodic cases and extensions to continuous-time chains, enhancing existing theoretical tools.

Findings

Derived two new norm-wise bounds for discrete-time Markov chains.

Extended bounds to continuous-time Markov chains using existing methods.

Provided examples illustrating the bounds and comparisons with prior results.

Abstract

In this paper, we are interested in investigating the perturbation bounds for the stationary distributions for discrete-time or continuous-time Markov chains on a countable state space. For discrete-time Markov chains, two new norm-wise bounds are obtained. The first bound is rather easy to be obtained since the needed condition, equivalent to uniform ergodicity, is imposed on the transition matrix directly. The second bound, which holds for a general (possibly periodic) Markov chain, involves finding a drift function. This drift function is closely related with the mean first hitting times. Some $V$-norm-wise bounds are also derived based on the results in Kartashov (1986). Moreover, we show how the bounds developed in this paper and one bound given in Seneta (1988) can be extended to continuous-time Markov chains. Several examples are shown to illustrate our results or to compare our bounds with the known ones in the literature.