Bayesian Higher Order Hidden Markov Models

TL;DR

This paper introduces a Bayesian nonparametric approach for higher order hidden Markov models that automatically identifies important lags and captures complex dependencies, improving modeling flexibility and performance.

Contribution

It proposes a tensor factorization-based Bayesian methodology for flexible higher order HMMs with automated lag selection and theoretical guarantees.

Findings

Outperforms existing higher order HMM methods in simulations

Effectively identifies relevant lags and interactions

Demonstrates practical utility on real data

Abstract

We consider the problem of flexible modeling of higher order hidden Markov models when the number of latent states and the nature of the serial dependence, including the true order, are unknown. We propose Bayesian nonparametric methodology based on tensor factorization techniques that can characterize any transition probability with a specified maximal order, allowing automated selection of the important lags and capturing higher order interactions among the lags. Theoretical results provide insights into identifiability of the emission distributions and asymptotic behavior of the posterior. We design efficient Markov chain Monte Carlo algorithms for posterior computation. In simulation experiments, the method vastly outperformed its first and higher order competitors not just in higher order settings, but, remarkably, also in first order cases. Practical utility is illustrated using real world applications.