Estimating Discrete Markov Models From Various Incomplete Data Schemes

TL;DR

This paper addresses the challenge of estimating transition probabilities in discrete Markov models when data are incomplete, proposing an adaptive MCMC algorithm to improve Bayesian inference efficiency.

Contribution

It introduces an adaptive MCMC method leveraging row dependence in transition matrices to accelerate Bayesian estimation with incomplete data.

Findings

The adaptive MCMC outperforms classical methods in convergence speed.

The proposed algorithm effectively handles various incomplete data schemes.

Empirical studies demonstrate improved inference accuracy.

Abstract

The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a case, the estimation of transition probabilities is straightforwardly made by counting one-step moves from a given state to another. In many real-life problems, however, the inference is much more difficult as state sequences are not fully observed, namely the state of each individual is known only for some given values of the time variable. A review of the problem is given, focusing on Monte Carlo Markov Chain (MCMC) algorithms to perform Bayesian inference and evaluate posterior distributions of the transition probabilities in this missing-data framework. Leaning on the dependence between the rows of the transition matrix, an adaptive MCMC mechanism accelerating the classical Metropolis-Hastings algorithm is then proposed and empirically studied.