Generalized Resilience and Robust Statistics

TL;DR

This paper extends robust statistics to general Wasserstein perturbations, introducing a resilient estimation framework that encompasses various data corruption types and simplifies existing methods for mean, regression, and covariance estimation.

Contribution

It generalizes the resilience concept to Wasserstein distances, providing new robust estimators with improved finite-sample rates and broad applicability.

Findings

Robust estimation is feasible under Wasserstein perturbations with moment or hypercontractive conditions.

The proposed estimators simplify and sometimes improve existing results in population and finite-sample contexts.

Connections are made to recent GAN-based robust estimation methods.

Abstract

Robust statistics traditionally focuses on outliers, or perturbations in total variation distance. However, a dataset could be corrupted in many other ways, such as systematic measurement errors and missing covariates. We generalize the robust statistics approach to consider perturbations under any Wasserstein distance, and show that robust estimation is possible whenever a distribution's population statistics are robust under a certain family of friendly perturbations. This generalizes a property called resilience previously employed in the special case of mean estimation with outliers. We justify the generalized resilience property by showing that it holds under moment or hypercontractive conditions. Even in the total variation case, these subsume conditions in the literature for mean estimation, regression, and covariance estimation; the resulting analysis simplifies and sometimes improves these known results in both population limit and finite-sample rate. Our robust estimators are based on minimum distance (MD) functionals (Donoho and Liu, 1988), which project onto a set of distributions under a discrepancy related to the perturbation. We present two approaches for designing MD estimators with good finite-sample rates: weakening the discrepancy and expanding the set of distributions. We also present connections to Gao et al. (2019)'s recent analysis of generative adversarial networks for robust estimation.