Bridging the Usability Gap: Theoretical and Methodological Advances for Spectral Learning of Hidden Markov Models

TL;DR

This paper advances spectral learning of hidden Markov models by providing asymptotic error analysis, proposing a new algorithm to reduce error propagation, and developing online variants for nonstationary data, improving robustness and efficiency.

Contribution

It introduces a novel projected spectral HMM algorithm and online learning methods, addressing error propagation and nonstationarity issues in spectral HMM estimation.

Findings

PSHMM offers more robust estimation than traditional spectral methods.

PSHMM retains computational efficiency of spectral learning.

Online variants adapt to nonstationary data effectively.

Abstract

The Baum-Welch (B-W) algorithm is the most widely accepted method for inferring hidden Markov models (HMM). However, it is prone to getting stuck in local optima, and can be too slow for many real-time applications. Spectral learning of HMMs (SHMM), based on the method of moments (MOM) has been proposed in the literature to overcome these obstacles. Despite its promises, asymptotic theory for SHMM has been elusive, and the long-run performance of SHMM can degrade due to unchecked propagation of error. In this paper, we (1) provide an asymptotic distribution for the approximate error of the likelihood estimated by SHMM, (2) propose a novel algorithm called projected SHMM (PSHMM) that mitigates the problem of error propagation, and (3) develop online learning variants of both SHMM and PSHMM that accommodate potential nonstationarity. We compare the performance of SHMM with PSHMM and estimation through the B-W algorithm on both simulated data and data from real world applications, and find that PSHMM not only retains the computational advantages of SHMM, but also provides more robust estimation and forecasting.