Robust-to-outliers square-root LASSO, simultaneous inference with a MOM approach

TL;DR

This paper introduces a robust MOM-based approach for simultaneous inference of regression functions and noise variance in outlier-corrupted data, extending to high-dimensional sparse linear models with adaptive estimation.

Contribution

It develops a novel MOM method for joint estimation of regression and noise variance, including a robust square-root LASSO extension and adaptive procedures for unknown sparsity.

Findings

Achieves minimax estimation rates under weak moment conditions.

Provides high-probability bounds for coefficient, noise, and sparsity level estimates.

Extends to adaptive estimation with unknown sparsity.

Abstract

We consider the least-squares regression problem with unknown noise variance, where the observed data points are allowed to be corrupted by outliers. Building on the median-of-means (MOM) method introduced by Lecue and Lerasle Ann.Statist.48(2):906-931(April 2020) in the case of known noise variance, we propose a general MOM approach for simultaneous inference of both the regression function and the noise variance, requiring only an upper bound on the noise level. Interestingly, this generalization requires care due to regularity issues that are intrinsic to the underlying convex-concave optimization problem. In the general case where the regression function belongs to a convex class, we show that our simultaneous estimator achieves with high probability the same convergence rates and a similar risk bound as if the noise level was unknown, as well as convergence rates for the estimated noise standard deviation. In the high-dimensional sparse linear setting, our estimator yields a robust analog of the square-root LASSO. Under weak moment conditions, it jointly achieves with high probability the minimax rates of estimation $s^{1/p} \sqrt{(1/n) \log(p/s)}$ for the $\ell_p$-norm of the coefficient vector, and the rate $\sqrt{(s/n) \log(p/s)}$ for the estimation of the noise standard deviation. Here $n$ denotes the sample size, $p$ the dimension and $s$ the sparsity level. We finally propose an extension to the case of unknown sparsity level $s$, providing a jointly adaptive estimator $(\widetilde \beta, \widetilde \sigma, \widetilde s)$. It simultaneously estimates the coefficient vector, the noise level and the sparsity level, with proven bounds on each of these three components that hold with high probability.