Nearly minimax robust estimator of the mean vector by iterative spectral dimension reduction

TL;DR

This paper introduces a spectral dimension reduction-based estimator for robustly estimating the mean of a sub-Gaussian distribution, achieving near-minimax optimal error bounds, high breakdown point, and computational efficiency.

Contribution

The paper proposes a novel SDR estimator that is robust, computationally efficient, and does not require prior knowledge of contamination rate, with theoretical guarantees close to the minimax bound.

Findings

Estimator achieves near-minimax optimal error bounds.

Breakdown point of the estimator is 1/2, the highest possible.

Computational complexity is manageable for high-dimensional data.

Abstract

We study the problem of robust estimation of the mean vector of a sub-Gaussian distribution. We introduce an estimator based on spectral dimension reduction (SDR) and establish a finite sample upper bound on its error that is minimax-optimal up to a logarithmic factor. Furthermore, we prove that the breakdown point of the SDR estimator is equal to $1/2$, the highest possible value of the breakdown point. In addition, the SDR estimator is equivariant by similarity transforms and has low computational complexity. More precisely, in the case of $n$ vectors of dimension $p$ -- at most $\varepsilon n$ out of which are adversarially corrupted -- the SDR estimator has a squared error of order $\big(\frac{r_\Sigma}{n} + \varepsilon^2\log(1/\varepsilon)\big){\log p}$ and a running time of order $p^3 + n p^2$. Here, $r_\Sigma\le p$ is the effective rank of the covariance matrix of the reference distribution. Another advantage of the SDR estimator is that it does not require knowledge of the contamination rate and does not involve sample splitting. We also investigate extensions of the proposed algorithm and of the obtained results in the case of (partially) unknown covariance matrix.