HMM for short independent sequences: Multiple sequence Baum-Welch application

TL;DR

This paper extends the Baum-Welch algorithm for Hidden Markov Models to handle multiple short, independent sequences, which is particularly useful for longitudinal population health studies with limited data per individual.

Contribution

It introduces pseudocode for training and decoding HMMs using multiple short sequences, adapting the classical approach to new data scenarios.

Findings

Effective training of HMMs with multiple short sequences demonstrated

Application to longitudinal health data shows practical relevance

Improved modeling of population trajectories with limited data

Abstract

In the classical setting, the training of a Hidden Markov Model (HMM) typically relies on a single, sufficiently long observation sequence that can be regarded as representative of the underlying stochastic process. In this context, the Expectation Maximization (EM) algorithm is applied in its specialized form for HMMs, namely the Baum Welch algorithm, which has been extensively employed in applications such as speech recognition. The objective of this work is to present pseudocode formulations for both the training and decoding procedures of HMMs in a different scenario, where the available data consist of multiple independent temporal sequences generated by the same model, each of relatively short duration, i.e., containing only a limited number of samples. Special emphasis is placed on the relevance of this formulation to longitudinal studies in population health, where datasets are naturally structured as collections of short trajectories across individuals with point data at follow up.