Robust Mean Estimation in High Dimensions: An Outlier Fraction Agnostic and Efficient Algorithm

TL;DR

This paper introduces a new robust mean estimation algorithm for high-dimensional data that is resistant to a significant fraction of outliers, does not require prior outlier fraction knowledge, and is computationally efficient.

Contribution

The paper formulates robust mean estimation as an $ ext{l}_p$-norm minimization problem, providing order-optimal solutions and an efficient algorithm that outperforms existing methods.

Findings

The algorithm achieves an outlier breakdown point of approximately 0.3.

It is computationally tractable with near-linear time complexity for $p=1$.

Experimental results show superior performance over state-of-the-art methods.

Abstract

The problem of robust mean estimation in high dimensions is studied, in which a certain fraction (less than half) of the datapoints can be arbitrarily corrupted. Motivated by compressive sensing, the robust mean estimation problem is formulated as the minimization of the $\ell_0$-`norm' of an \emph{outlier indicator vector}, under a second moment constraint on the datapoints. The $\ell_0$-`norm' is then relaxed to the $\ell_p$-norm ($0<p\leq 1$) in the objective, and it is shown that the global minima for each of these objectives are order-optimal and have optimal breakdown point for the robust mean estimation problem. Furthermore, a computationally tractable iterative $\ell_p$-minimization and hard thresholding algorithm is proposed that outputs an order-optimal robust estimate of the population mean. The proposed algorithm (with breakdown point $\approx 0.3$) does not require prior knowledge of the fraction of outliers, in contrast with most existing algorithms, and for $p=1$ it has near-linear time complexity. Both synthetic and real data experiments demonstrate that the proposed algorithm outperforms state-of-the-art robust mean estimation methods.