An EM Algorithm for Continuous-time Bivariate Markov Chains

TL;DR

This paper introduces an EM algorithm for estimating the generator of continuous-time bivariate Markov chains with only one observable process, generalizing existing models and avoiding complex numerical schemes.

Contribution

It develops a novel EM algorithm for parameter estimation in continuous-time bivariate Markov chains without assuming Markovian structure or requiring numerical integration.

Findings

The EM algorithm accurately estimates the generator from observable data.

The method applies to multivariate Markov chains.

It outperforms existing estimation techniques in efficiency.

Abstract

We study properties and parameter estimation of finite-state homogeneous continuous-time bivariate Markov chains. Only one of the two processes of the bivariate Markov chain is observable. The general form of the bivariate Markov chain studied here makes no assumptions on the structure of the generator of the chain, and hence, neither the underlying process nor the observable process is necessarily Markov. The bivariate Markov chain allows for simultaneous jumps of the underlying and observable processes. Furthermore, the inter-arrival time of observed events is phase-type. The bivariate Markov chain generalizes the batch Markovian arrival process as well as the Markov modulated Markov process. We develop an expectation-maximization (EM) procedure for estimating the generator of a bivariate Markov chain, and we demonstrate its performance. The procedure does not rely on any numerical integration or sampling scheme of the continuous-time bivariate Markov chain. The proposed EM algorithm is equally applicable to multivariate Markov chains.