GTH Algorithm, Censored Markov Chains, and $RG$-Factorization in Block-Form

TL;DR

This paper explores the block-form GTH algorithm's connection to censored Markov chains and RG-factorization, extending it to infinite states and proposing an asymptotically optimal approximation method.

Contribution

It links the block-form GTH algorithm with RG-factorization and censored chains, extending its applicability to infinite-state Markov chains and introducing a new approximation technique.

Findings

The block-form GTH algorithm is equivalent to solving a system using RG-factorization.

Extension of the GTH algorithm to infinite-state Markov chains is validated.

The proposed RA-CM method provides an asymptotically optimal stationary distribution approximation.

Abstract

In 1985, Grassmann, Taksar, and Heyman published their celebrated paper, in which they introduced a numerically stable algorithm for computing the stationary probabilities of a finite-state Markov chain, one of the key performance quantities in both theory and applications. This algorithm later became the well-known GTH algorithm (or the state-reduction method) in the literature, becoming one of the standard algorithms in applied probability. Later, this algorithm was extended to deal with the stationary distributions of block-structured Markov chains with repeating rows. In this paper, we focus on the block-form GTH algorithm and organize it into two parts. In the first part, we connect the block-form GTH algorithm to censored Markov chains and the block-form $RG$-factorization. We show that the forward block-elimination and back block-form substitution of the block-form GTH algorithm are equivalent to solving a system formulated using the $RG$-factorization in two steps. We also show that this connection remains valid when the block-form GTH algorithm is extended to infinite-state Markov chains. It is well known that censoring an infinite-state Markov chain to a finite state space yields a stationary distribution that provides a best approximation to the stationary distribution of the original infinite-state Markov chain. In the second part, we first derive an explicit expression for the censored Markov chain from the infinite state space to a finite space for Markov chains of $M/G/1$ type. Based on this expression, we propose a renormalized approximated censored transition matrix (RA-CM). The resulting stationary distribution is shown to be asymptotically optimal in terms of approximation error. We compare the approximation error of the RA-CM with the error arising from the last-block-column augmentation.