Maximum likelihood estimation of hidden Markov models for continuous longitudinal data with missing responses and dropout

TL;DR

This paper introduces a maximum likelihood estimation method for hidden Markov models tailored for continuous longitudinal data with various missing data patterns, including dropout, using an EM algorithm.

Contribution

It extends hidden Markov models to handle multiple missing data patterns and dropout in longitudinal data under the MAR assumption with an EM algorithm.

Findings

The method effectively estimates model parameters with missing data.

Simulation studies demonstrate accurate parameter recovery.

Application to medical data shows practical utility.

Abstract

We propose an inferential approach for maximum likelihood estimation of the hidden Markov models for continuous responses. We extend to the case of longitudinal observations the finite mixture model of multivariate Gaussian distributions with Missing At Random (MAR) outcomes, also accounting for possible dropout. The resulting hidden Markov model accounts for different types of missing pattern: (i) partially missing outcomes at a given time occasion; (ii) completely missing outcomes at a given time occasion (intermittent pattern); (iii) dropout before the end of the period of observation (monotone pattern). The MAR assumption is formulated to deal with the first two types of missingness, while to account for informative dropout we assume an extra absorbing state. Maximum likelihood estimation of the model parameters is based on an extended Expectation-Maximization algorithm relying on suitable recursions. The proposal is illustrated by a Monte Carlo simulation study and an application based on historical data on primary biliary cholangitis.