Augmented truncation approximations to the solution of Poisson's equation for Markov chains

TL;DR

This paper introduces augmented truncation approximation methods for solving Poisson's equation in Markov chains, providing convergence analysis and applications to birth-death processes.

Contribution

It develops a new computational framework using augmented truncation for solving Poisson's equation in infinite-state Markov chains, with convergence proofs.

Findings

Convergence established under monotonicity assumptions.

Applicable to both DTMCs and CTMCs.

Extended to variance approximation in CLTs.

Abstract

Poisson's equation has a lot of applications in various areas. Usually it is hard to derive the explicit expression of the solution of Poisson's equation for a Markov chain on an infinitely many state space. We will present a computational framework for the solution for both discrete-time Markov chains (DTMCs) and continuous-time Markov chains (CTMCs), by developing the technique of augmented truncation approximations. The convergence to the solution is investigated in terms of the assumption about the monotonicity of the first return times, and is further established for two types of truncation approximation schemes: the censored chain and the linear augmented truncation. Moreover, truncation approximations to the variance constant in central limit theorems (CLTs) are also considered. The results obtained are applied to discrete-time single-birth processes and continuous-time single-death processes.