Outlier-Robust High-Dimensional Sparse Estimation via Iterative Filtering

TL;DR

This paper introduces practical, efficient algorithms for robust high-dimensional sparse estimation tasks, effectively handling adversarial data corruption using spectral techniques, and demonstrating superior performance over previous methods.

Contribution

The paper presents the first practical algorithms for robust sparse mean estimation and sparse PCA that are both computationally efficient and nearly optimal in robustness guarantees.

Findings

Algorithms are scalable and outperform previous approaches on synthetic data.

Methods nearly match the best error rates in the absence of corruptions.

Spectral techniques effectively remove outliers iteratively.

Abstract

We study high-dimensional sparse estimation tasks in a robust setting where a constant fraction of the dataset is adversarially corrupted. Specifically, we focus on the fundamental problems of robust sparse mean estimation and robust sparse PCA. We give the first practically viable robust estimators for these problems. In more detail, our algorithms are sample and computationally efficient and achieve near-optimal robustness guarantees. In contrast to prior provable algorithms which relied on the ellipsoid method, our algorithms use spectral techniques to iteratively remove outliers from the dataset. Our experimental evaluation on synthetic data shows that our algorithms are scalable and significantly outperform a range of previous approaches, nearly matching the best error rate without corruptions.