A Revisit of Block Power Methods for Finite State Markov Chain Applications

TL;DR

This paper analyzes and enhances block power methods for efficiently computing the stationary distribution of Markov chains, especially when the eigenvalues are closely spaced, by introducing a sliding window scheme to reduce computational costs.

Contribution

The paper revisits block power methods, demonstrating their effectiveness based on eigenvalue separation, and introduces a sliding window scheme to further reduce computational costs in Markov chain applications.

Findings

Block power method convergence depends on the $(s+1)$th eigenvalue.

The sliding window scheme accelerates convergence by removing influences of smaller eigenvalues.

The combined approach reduces matrix-vector multiplications in large Markov chains.

Abstract

In this paper, we revisit the generalized block power methods for approximating the eigenvector associated with $\lambda_1 = 1$ of a Markov chain transition matrix. Our analysis of the block power method shows that when $s$ linearly independent probability vectors are used as the initial block, the convergence of the block power method to the stationary distribution depends on the magnitude of the $(s+1)$th dominant eigenvalue $\lambda_{s+1}$ of $P$ instead of that of $\lambda_2$ in the power method. Therefore, the block power method with block size $s$ is particularly effective for transition matrices where $|\lambda_{s+1}|$ is well separated from $\lambda_1 = 1$ but $|\lambda_2|$ is not. This approach is particularly useful when visiting the elements of a large transition matrix is the main computational bottleneck over matrix--vector multiplications, where the block power method can effectively reduce the total number of times to pass over the matrix. To further reduce the overall computational cost, we combine the block power method with a sliding window scheme, taking advantage of the subsequent vectors of the latest $s$ iterations to assemble the block matrix. The sliding window scheme correlates vectors in the sliding window to quickly remove the influences from the eigenvalues whose magnitudes are smaller than $|\lambda_{s}|$ to reduce the overall number of matrix--vector multiplications to reach convergence. Finally, we compare the effectiveness of these methods in a Markov chain model representing a stochastic luminal calcium release site.