Exact Estimation for Markov Chain Equilibrium Expectations

TL;DR

This paper introduces exact estimation algorithms for Markov chain equilibrium expectations, offering unbiased estimators for functionals on certain classes of Markov chains, with theoretical advantages over exact simulation.

Contribution

The paper develops a new class of Monte Carlo methods called exact estimation algorithms for unbiased equilibrium expectation estimation in Markov chains.

Findings

Algorithms applicable to positive Harris recurrent chains

Methods for chains contracting on average

Provides theoretical advantages over exact simulation

Abstract

We introduce a new class of Monte Carlo methods, which we call exact estimation algorithms. Such algorithms provide unbiased estimators for equilibrium expectations associated with real- valued functionals defined on a Markov chain. We provide easily implemented algorithms for the class of positive Harris recurrent Markov chains, and for chains that are contracting on average. We further argue that exact estimation in the Markov chain setting provides a significant theoretical relaxation relative to exact simulation methods.