Bayesian analysis of variable-order, reversible Markov chains

TL;DR

This paper introduces a conjugate prior for reversible Markov chains of fixed and variable order, facilitating order testing and parameter estimation in such models.

Contribution

It develops a novel conjugate prior based on reinforced random walks for reversible Markov chains, including variable-order extensions.

Findings

The prior enables effective testing of the Markov chain order.

It allows for accurate parameter estimation in reversible Markov models.

The approach generalizes to variable-order chains.

Abstract

We define a conjugate prior for the reversible Markov chain of order $r$. The prior arises from a partially exchangeable reinforced random walk, in the same way that the Beta distribution arises from the exchangeable Poly\'{a} urn. An extension to variable-order Markov chains is also derived. We show the utility of this prior in testing the order and estimating the parameters of a reversible Markov model.