Agnostic Estimation of Mean and Covariance

TL;DR

This paper introduces polynomial-time algorithms for robustly estimating the mean and covariance of high-dimensional distributions from data with a fraction of malicious noise, achieving near-optimal error bounds.

Contribution

It provides the first polynomial-time algorithms with provable guarantees for agnostic estimation of mean and covariance under adversarial noise.

Findings

Algorithms achieve error bounds close to information-theoretic limits.

Applicable to Gaussian parameters, mixtures, ICA, and SVD.

Demonstrates robustness in high-dimensional, adversarial settings.

Abstract

We consider the problem of estimating the mean and covariance of a distribution from iid samples in $\mathbb{R}^n$, in the presence of an $\eta$ fraction of malicious noise; this is in contrast to much recent work where the noise itself is assumed to be from a distribution of known type. The agnostic problem includes many interesting special cases, e.g., learning the parameters of a single Gaussian (or finding the best-fit Gaussian) when $\eta$ fraction of data is adversarially corrupted, agnostically learning a mixture of Gaussians, agnostic ICA, etc. We present polynomial-time algorithms to estimate the mean and covariance with error guarantees in terms of information-theoretic lower bounds. As a corollary, we also obtain an agnostic algorithm for Singular Value Decomposition.