All-In-One Robust Estimator of the Gaussian Mean

TL;DR

This paper introduces a single, computationally efficient robust estimator for the mean of multivariate Gaussian distributions that maintains multiple desirable statistical properties even under adversarial contamination.

Contribution

It proposes a novel iterative reweighting estimator that is robust, efficient, and equivariant, with theoretical guarantees including minimax optimality and a non-asymptotic risk bound.

Findings

Achieves high breakdown point of 0.5 and nearly-minimax-rate-breakdown point of 0.28.

Provides a dimension-free non-asymptotic risk bound involving the effective rank.

Extends to sub-Gaussian distributions and unknown parameters.

Abstract

The goal of this paper is to show that a single robust estimator of the mean of a multivariate Gaussian distribution can enjoy five desirable properties. First, it is computationally tractable in the sense that it can be computed in a time which is at most polynomial in dimension, sample size and the logarithm of the inverse of the contamination rate. Second, it is equivariant by translations, uniform scaling and orthogonal transformations. Third, it has a high breakdown point equal to $0.5$, and a nearly-minimax-rate-breakdown point approximately equal to $0.28$. Fourth, it is minimax rate optimal, up to a logarithmic factor, when data consists of independent observations corrupted by adversarially chosen outliers. Fifth, it is asymptotically efficient when the rate of contamination tends to zero. The estimator is obtained by an iterative reweighting approach. Each sample point is assigned a weight that is iteratively updated by solving a convex optimization problem. We also establish a dimension-free non-asymptotic risk bound for the expected error of the proposed estimator. It is the first result of this kind in the literature and involves only the effective rank of the covariance matrix. Finally, we show that the obtained results can be extended to sub-Gaussian distributions, as well as to the cases of unknown rate of contamination or unknown covariance matrix.