A Regeneration-based a Posteriori Error Bound for a Markov Chain Stationary Distribution Truncation Algorithm

TL;DR

This paper introduces a new, tighter a posteriori error bound for approximating the stationary distribution of large or infinite Markov chains, leveraging regenerative structure and Lyapunov functions.

Contribution

It develops a novel regenerative-based a posteriori error bound that is easier to compute and tighter than existing a priori bounds for Markov chain stationary distribution truncation.

Findings

The bound exploits regenerative structure and Lyapunov functions.

It decomposes the chain into excursions for error estimation.

The bound is computationally straightforward and does not require linear programming.

Abstract

When the state space of a discrete state space positive recurrent Markov chain is infinite or very large, it becomes necessary to truncate the state space in order to facilitate numerical computation of the stationary distribution. This paper develops a new approach for bounding the truncation error that arises when computing approximations to the stationary distribution. This rigorous a posteriori error bound exploits the regenerative structure of the chain and assumes knowledge of a Lyapunov function. Because the bound is a posteriori (and leverages the computations done to calculate the stationary distribution itself), it tends to be much tighter than a priori bounds. The bound decomposes the regenerative cycle into a random number of excursions from a set $K$ defined in terms of the Lyapunov function into the complement of the truncation set $A$. The bound can be easily computed, and does not (for example) involve a linear program, as do some other error bounds.