Bayesian nonparametric analysis of reversible Markov chains

TL;DR

This paper introduces a flexible Bayesian nonparametric prior for reversible Markov chains, enabling improved inference in complex systems like protein folding, by generalizing existing reinforced random walk models with a new three-parameter scheme.

Contribution

It develops a novel three-parameter random walk scheme that unifies and extends existing models, providing a new nonparametric prior for Bayesian analysis of reversible Markov chains.

Findings

The scheme smoothly interpolates between reinforced random walk and Hoppe urn models.

Applied to molecular dynamics data, it effectively infers transition kernels.

Demonstrates improved modeling of state visitation in complex systems.

Abstract

We introduce a three-parameter random walk with reinforcement, called the $(\theta,\alpha,\beta)$ scheme, which generalizes the linearly edge reinforced random walk to uncountable spaces. The parameter $\beta$ smoothly tunes the $(\theta,\alpha,\beta)$ scheme between this edge reinforced random walk and the classical exchangeable two-parameter Hoppe urn scheme, while the parameters $\alpha$ and $\theta$ modulate how many states are typically visited. Resorting to de Finetti's theorem for Markov chains, we use the $(\theta,\alpha,\beta)$ scheme to define a nonparametric prior for Bayesian analysis of reversible Markov chains. The prior is applied in Bayesian nonparametric inference for species sampling problems with data generated from a reversible Markov chain with an unknown transition kernel. As a real example, we analyze data from molecular dynamics simulations of protein folding.