Learning HMMs with Nonparametric Emissions via Spectral Decompositions of Continuous Matrices

TL;DR

This paper introduces a spectral method for learning nonparametric hidden Markov models with smooth emission densities, leveraging continuous linear algebra and polynomial approximations for efficient estimation.

Contribution

It develops a novel spectral algorithm that estimates nonparametric HMMs using continuous matrices and SVD, with theoretical guarantees and practical efficiency.

Findings

Competitive performance on synthetic and real data

Sample complexity bounds established for nonparametric density estimation

Efficient implementation via Chebyshev polynomial approximations

Abstract

Recently, there has been a surge of interest in using spectral methods for estimating latent variable models. However, it is usually assumed that the distribution of the observations conditioned on the latent variables is either discrete or belongs to a parametric family. In this paper, we study the estimation of an $m$-state hidden Markov model (HMM) with only smoothness assumptions, such as H\"olderian conditions, on the emission densities. By leveraging some recent advances in continuous linear algebra and numerical analysis, we develop a computationally efficient spectral algorithm for learning nonparametric HMMs. Our technique is based on computing an SVD on nonparametric estimates of density functions by viewing them as \emph{continuous matrices}. We derive sample complexity bounds via concentration results for nonparametric density estimation and novel perturbation theory results for continuous matrices. We implement our method using Chebyshev polynomial approximations. Our method is competitive with other baselines on synthetic and real problems and is also very computationally efficient.