Bayesian inference for continuous-time hidden Markov models with an unknown number of states

TL;DR

This paper develops a Bayesian inference method for continuous-time hidden Markov models with an unknown number of states, using reversible jump MCMC and a novel split-combine move, applicable to clustering and real healthcare data.

Contribution

It introduces an efficient reversible jump MCMC algorithm with a split-combine move for continuous-time models with unknown states, extending to clustering applications.

Findings

Effective exploration of parameter space demonstrated in simulations.

Successful application to healthcare data from Quebec.

Method outperforms existing approaches in continuous-time settings.

Abstract

We consider the modeling of data generated by a latent continuous-time Markov jump process with a state space of finite but unknown dimensions. Typically in such models, the number of states has to be pre-specified, and Bayesian inference for a fixed number of states has not been studied until recently. In addition, although approaches to address the problem for discrete-time models have been developed, no method has been successfully implemented for the continuous-time case. We focus on reversible jump Markov chain Monte Carlo which allows the trans-dimensional move among different numbers of states in order to perform Bayesian inference for the unknown number of states. Specifically, we propose an efficient split-combine move which can facilitate the exploration of the parameter space, and demonstrate that it can be implemented effectively at scale. Subsequently, we extend this algorithm to the context of model-based clustering, allowing numbers of states and clusters both determined during the analysis. The model formulation, inference methodology, and associated algorithm are illustrated by simulation studies. Finally, We apply this method to real data from a Canadian healthcare system in Quebec.