An Expectation-Maximization Algorithm for Continuous-time Hidden Markov Models

TL;DR

This paper introduces a unified continuous-time hidden Markov model framework, extending inference methods to continuous settings with jump and diffusion processes, and develops an EM algorithm with Monte Carlo sampling for parameter estimation.

Contribution

It extends classical HMM inference to continuous-time models, deriving differential equations for hidden states and continuous EM formulas, with a Monte Carlo approach for practical estimation.

Findings

Derived differential equations for hidden states in continuous time.

Formulated continuous-time EM algorithm for parameter estimation.

Proposed Monte Carlo sampling method for hidden state posterior.

Abstract

We propose a unified framework that extends the inference methods for classical hidden Markov models to continuous settings, where both the hidden states and observations occur in continuous time. Two different settings are analyzed: hidden jump process with a finite state space, and hidden diffusion process with a continuous state space. For each setting, we first estimate the hidden states given the observations and model parameters, showing that the posterior distribution of the hidden states can be described by differential equations in continuous time. We then consider the estimation of unknown model parameters, deriving the continuous-time formulas for the expectation-maximization algorithm. We also propose a Monte Carlo method based on the continuous formulation, sampling the posterior distribution of the hidden states and updating the parameter estimation.