Robust Estimation of Covariance Matrices: Adversarial Contamination and Beyond

TL;DR

This paper develops a robust covariance matrix estimator effective in high-dimensional, contaminated, or heavy-tailed data scenarios, with theoretical guarantees and efficient algorithms.

Contribution

It introduces an adaptive estimator for low-rank covariance matrices robust to adversarial contamination and heavy tails, with tight deviation bounds and practical algorithms.

Findings

Estimator achieves tight deviation guarantees under weak assumptions.

Method adapts to low-rank structure and contamination proportion.

Algorithms enable efficient approximation of the estimator.

Abstract

We consider the problem of estimating the covariance structure of a random vector $Y\in \mathbb R^d$ from a sample $Y_1,\ldots,Y_n$. We are interested in the situation when $d$ is large compared to $n$ but the covariance matrix $\Sigma$ of interest has (exactly or approximately) low rank. We assume that the given sample is (a) $\epsilon$-adversarially corrupted, meaning that $\epsilon$ fraction of the observations could have been replaced by arbitrary vectors, or that (b) the sample is i.i.d. but the underlying distribution is heavy-tailed, meaning that the norm of $Y$ possesses only finite fourth moments. We propose an estimator that is adaptive to the potential low-rank structure of the covariance matrix as well as to the proportion of contaminated data, and admits tight deviation guarantees despite rather weak assumptions on the underlying distribution. Finally, we discuss the algorithms that allow to approximate the proposed estimator in a numerically efficient way.