On approximating the stationary distribution of time-reversible Markov chains

TL;DR

This paper introduces a new randomized algorithm for approximating the stationary distribution of time-reversible Markov chains more efficiently, overcoming the limitations of existing methods that depend heavily on the target state's probability.

Contribution

The authors present a simple randomized algorithm that surpasses the traditional small-$\pi(v)$ barrier for time-reversible Markov chains, improving approximation efficiency.

Findings

Breaks the small-$\pi(v)$ barrier in approximation complexity

Provides a more efficient method for stationary distribution estimation

Applicable to time-reversible Markov chains

Abstract

Approximating the stationary probability of a state in a Markov chain through Markov chain Monte Carlo techniques is, in general, inefficient. Standard random walk approaches require $\tilde{O}(\tau/\pi(v))$ operations to approximate the probability $\pi(v)$ of a state $v$ in a chain with mixing time $\tau$, and even the best available techniques still have complexity $\tilde{O}(\tau^{1.5}/\pi(v)^{0.5})$, and since these complexities depend inversely on $\pi(v)$, they can grow beyond any bound in the size of the chain or in its mixing time. In this paper we show that, for time-reversible Markov chains, there exists a simple randomized approximation algorithm that breaks this "small-$\pi(v)$ barrier".