Adaptive Bayesian Inference of Markov Transition Rates

TL;DR

This paper presents an adaptive Bayesian method for efficiently estimating transition rates in Markov chains, reducing sampling variance and error through sequential, optimal experimental design that adapts after each observation.

Contribution

It introduces a sequential Bayesian optimal design approach for Markov chain parameter estimation that can be extended beyond simple two-state models.

Findings

Significantly reduces variance in parameter estimates.

Achieves lower overall error across diverse Markov models.

Efficiently determines optimal sampling times for each observation.

Abstract

Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past observations, adaptive approaches adjust sampling constraints online as model parameter estimates are refined, continually maximizing expected information gained or variance reduced. We apply adaptive Bayesian inference to estimate transition rates of Markov chains, a common class of models for stochastic processes in nature. Unlike most previous studies, our sequential Bayesian optimal design is updated with each observation, and can be simply extended beyond two-state models to birth-death processes and multistate models. By iteratively finding the best time to obtain each sample, our adaptive algorithm maximally reduces variance, resulting in lower overall error in ground truth parameter estimates across a wide range of Markov chain parameterizations and conformations.