# High Dimensional Robust $M$-Estimation: Arbitrary Corruption and Heavy Tails

## TL;DR

This paper introduces a flexible framework for high-dimensional $M$-estimation under heavy tails and arbitrary corruptions, providing new robust algorithms with optimal statistical guarantees.

## Contribution

The paper defines the Robust Descent Condition (RDC) and shows that it enables robust, minimax-optimal $M$-estimation algorithms in heavy-tailed and corrupted data scenarios.

## Key findings

- Median-of-means gradient estimator satisfies RDC for heavy tails.
- Trimmed gradient estimator satisfies RDC for arbitrary corruptions.
- Robust Hard Thresholding achieves minimax optimal rates in tested scenarios.

## Abstract

We consider the problem of sparsity-constrained $M$-estimation when both explanatory and response variables have heavy tails (bounded 4-th moments), or a fraction of arbitrary corruptions. We focus on the $k$-sparse, high-dimensional regime where the number of variables $d$ and the sample size $n$ are related through $n \sim k \log d$. We define a natural condition we call the Robust Descent Condition (RDC), and show that if a gradient estimator satisfies the RDC, then Robust Hard Thresholding (IHT using this gradient estimator), is guaranteed to obtain good statistical rates. The contribution of this paper is in showing that this RDC is a flexible enough concept to recover known results, and obtain new robustness results. Specifically, new results include: (a) For $k$-sparse high-dimensional linear- and logistic-regression with heavy tail (bounded 4-th moment) explanatory and response variables, a linear-time-computable median-of-means gradient estimator satisfies the RDC, and hence Robust Hard Thresholding is minimax optimal; (b) When instead of heavy tails we have $O(1/\sqrt{k}\log(nd))$-fraction of arbitrary corruptions in explanatory and response variables, a near linear-time computable trimmed gradient estimator satisfies the RDC, and hence Robust Hard Thresholding is minimax optimal. We demonstrate the effectiveness of our approach in sparse linear, logistic regression, and sparse precision matrix estimation on synthetic and real-world US equities data.