Clustering Time Series and the Surprising Robustness of HMMs

TL;DR

This paper demonstrates that maximum likelihood HMM estimators are surprisingly effective in approximating source distributions in a broad class of non-stationary time series models where sources change infrequently.

Contribution

It introduces a general non-stationary model for time series and proves that HMM estimators can accurately recover second moments under this model, extending their applicability.

Findings

HMM estimators recover the correct second moment in non-stationary models.

The approach extends to higher moments beyond the second.

HMMs are robust even when data lacks strict Markov or stationarity properties.

Abstract

Suppose that we are given a time series where consecutive samples are believed to come from a probabilistic source, that the source changes from time to time and that the total number of sources is fixed. Our objective is to estimate the distributions of the sources. A standard approach to this problem is to model the data as a hidden Markov model (HMM). However, since the data often lacks the Markov or the stationarity properties of an HMM, one can ask whether this approach is still suitable or perhaps another approach is required. In this paper we show that a maximum likelihood HMM estimator can be used to approximate the source distributions in a much larger class of models than HMMs. Specifically, we propose a natural and fairly general non-stationary model of the data, where the only restriction is that the sources do not change too often. Our main result shows that for this model, a maximum-likelihood HMM estimator produces the correct second moment of the data, and the results can be extended to higher moments.