Robust Generalized Method of Moments: A Finite Sample Viewpoint

TL;DR

This paper introduces a computationally efficient robust GMM estimator that can tolerate a constant fraction of adversarial outliers, providing strong recovery guarantees and outperforming classical methods in contaminated datasets.

Contribution

The paper develops the first efficient robust GMM estimator with provable guarantees under adversarial corruption, extending robust statistics techniques to GMM.

Findings

Estimator tolerates a constant fraction of outliers

Provides an $ ext{O}(\sqrt{ ext{outlier fraction}})$ recovery guarantee

Outperforms classical IV and Huber regression in experiments

Abstract

For many inference problems in statistics and econometrics, the unknown parameter is identified by a set of moment conditions. A generic method of solving moment conditions is the Generalized Method of Moments (GMM). However, classical GMM estimation is potentially very sensitive to outliers. Robustified GMM estimators have been developed in the past, but suffer from several drawbacks: computational intractability, poor dimension-dependence, and no quantitative recovery guarantees in the presence of a constant fraction of outliers. In this work, we develop the first computationally efficient GMM estimator (under intuitive assumptions) that can tolerate a constant $\epsilon$ fraction of adversarially corrupted samples, and that has an $\ell_2$ recovery guarantee of $O(\sqrt{\epsilon})$. To achieve this, we draw upon and extend a recent line of work on algorithmic robust statistics for related but simpler problems such as mean estimation, linear regression and stochastic optimization. As two examples of the generality of our algorithm, we show how our estimation algorithm and assumptions apply to instrumental variables linear and logistic regression. Moreover, we experimentally validate that our estimator outperforms classical IV regression and two-stage Huber regression on synthetic and semi-synthetic datasets with corruption.