Sample Size Considerations for Bayesian Multilevel Hidden Markov Models: A Simulation Study on Multivariate Continuous Data with highly overlapping Component Distributions based on Sleep Data

TL;DR

This study investigates how sample size, number of occasions, and variability affect Bayesian multilevel Hidden Markov Model estimates using sleep data, providing practical guidelines for researchers in social and behavioral sciences.

Contribution

It offers the first systematic simulation-based evaluation of sample size requirements for Bayesian multilevel HMMs with multivariate continuous data.

Findings

Number of subjects significantly impacts model performance.

Number of occasions influences the accuracy of latent state transition estimates.

Data characteristics affect parameter estimation and model reliability.

Abstract

Spurred in part by the ever-growing number of sensors and web-based methods of collecting data, the use of Intensive Longitudinal Data (ILD) is becoming more common in the social and behavioural sciences. The ILD collected in this field are often hypothesised to be the result of latent states (e.g. behaviour, emotions), and the promise of ILD lies in its ability to capture the dynamics of these states as they unfold in time. In particular, by collecting data for multiple subjects, researchers can observe how such dynamics differ between subjects. The Bayesian Multilevel Hidden Markov Model (mHMM) is a relatively novel model that is suited to model the ILD of this kind while taking into account heterogeneity between subjects. While the mHMM has been applied in a variety of settings, large-scale studies that examine the required sample size for this model are lacking. In this paper, we address this research gap by conducting a simulation study to evaluate the effect of changing (1) the number of subjects, (2) the number of occasions, and (3) the between subjects variability on parameter estimates obtained by the mHMM. We frame this simulation study in the context of sleep research, which consists of multivariate continuous data that displays considerable overlap in the state dependent component distributions. In addition, we generate a set of baseline scenarios with more general data properties. Overall, the number of subjects has the largest effect on model performance. However, the number of occasions is important to adequately model latent state transitions. We discuss how the characteristics of the data influence parameter estimation and provide recommendations to researchers seeking to apply the mHMM to their own data.