Spectral M-estimation with Applications to Hidden Markov Models

TL;DR

This paper introduces a new M-estimation framework for spectral methods that improves sample efficiency and regularization in hidden Markov models, outperforming traditional moment estimators.

Contribution

It develops a generalized M-estimator that achieves optimal sample efficiency and integrates regularization into spectral methods for hidden Markov models.

Findings

Achieves optimal sample efficiency rates for moment-based estimators.

Demonstrates improved prediction accuracy with the proposed method.

Shows empirical gains in sample efficiency on hidden Markov models.

Abstract

Method of moment estimators exhibit appealing statistical properties, such as asymptotic unbiasedness, for nonconvex problems. However, they typically require a large number of samples and are extremely sensitive to model misspecification. In this paper, we apply the framework of M-estimation to develop both a generalized method of moments procedure and a principled method for regularization. Our proposed M-estimator obtains optimal sample efficiency rates (in the class of moment-based estimators) and the same well-known rates on prediction accuracy as other spectral estimators. It also makes it straightforward to incorporate regularization into the sample moment conditions. We demonstrate empirically the gains in sample efficiency from our approach on hidden Markov models.