Estimation and uncertainty of reversible Markov models

TL;DR

This paper introduces new algorithms for estimating reversible Markov transition matrices from simulation data, including maximum likelihood and Bayesian methods, with implementations in the PyEMMA software to improve molecular kinetics modeling.

Contribution

It presents novel algorithms for reversible transition matrix estimation, incorporating Bayesian inference and implementation in PyEMMA, enhancing analysis of molecular kinetics.

Findings

Developed a new algorithm for reversible transition matrix estimation.

Implemented Bayesian posterior inference methods for reversible Markov models.

Provided software tools in PyEMMA for practical application.

Abstract

Reversibility is a key concept in Markov models and Master-equation models of molecular kinetics. The analysis and interpretation of the transition matrix encoding the kinetic properties of the model relies heavily on the reversibility property. The estimation of a reversible transition matrix from simulation data is therefore crucial to the successful application of the previously developed theory. In this work we discuss methods for the maximum likelihood estimation of transition matrices from finite simulation data and present a new algorithm for the estimation if reversibility with respect to a given stationary vector is desired. We also develop new methods for the Bayesian posterior inference of reversible transition matrices with and without given stationary vector taking into account the need for a suitable prior distribution preserving the meta- stable features of the observed process during posterior inference. All algorithms here are implemented in the PyEMMA software - http://pyemma.org - as of version 2.0.