Robust and Tuning-Free Sparse Linear Regression via Square-Root Slope

TL;DR

This paper introduces a robust, tuning-free sparse linear regression method called Square-Root Slope, which is designed to handle adversarial data contamination, heavy-tailed noise, and unknown sparsity levels with provable guarantees.

Contribution

It proposes a novel, adaptive, and computationally tractable Square-Root Slope estimator that achieves near-optimal performance under adversarial and heavy-tailed noise conditions.

Findings

Estimator is robust to adversarial contamination.

Performance guarantees match lower bounds up to logarithmic factors.

Method is fully adaptive and computationally feasible.

Abstract

We consider the high-dimensional linear regression model and assume that a fraction of the measurements are altered by an adversary with complete knowledge of the data and the underlying distribution. We are interested in a scenario where dense additive noise is heavy-tailed while the measurement vectors follow a sub-Gaussian distribution. Within this framework, we establish minimax lower bounds for the performance of an arbitrary estimator that depend on the the fraction of corrupted observations as well as the tail behavior of the additive noise. Moreover, we design a modification of the so-called Square-Root Slope estimator with several desirable features: (a) it is provably robust to adversarial contamination, and satisfies performance guarantees in the form of sub-Gaussian deviation inequalities that match the lower error bounds, up to logarithmic factors; (b) it is fully adaptive with respect to the unknown sparsity level and the variance of the additive noise, and (c) it is computationally tractable as a solution of a convex optimization problem. To analyze performance of the proposed estimator, we prove several properties of matrices with sub-Gaussian rows that may be of independent interest.