Correct Approximation of Stationary Distributions

TL;DR

This paper introduces a new method for accurately approximating stationary distributions of Markov chains, overcoming scalability issues and avoiding incorrect results from naive approaches, by leveraging recent advances in partial exploration and mean payoff computation.

Contribution

The paper presents a novel approach that ensures correct, converging approximations of stationary distributions using recent techniques in partial exploration and mean payoff calculation.

Findings

The new method provides accurate approximations where naive methods fail.

It scales better to large Markov chains.

The approach guarantees convergence to the true stationary distribution.

Abstract

A classical problem for Markov chains is determining their stationary (or steady-state) distribution. This problem has an equally classical solution based on eigenvectors and linear equation systems. However, this approach does not scale to large instances, and iterative solutions are desirable. It turns out that a naive approach, as used by current model checkers, may yield completely wrong results. We present a new approach, which utilizes recent advances in partial exploration and mean payoff computation to obtain a correct, converging approximation.