Support estimation in high-dimensional heteroscedastic mean regression

TL;DR

This paper develops a robust high-dimensional support estimation method for heteroscedastic, heavy-tailed linear regression models using a smooth Huber loss and adaptive LASSO, achieving sign consistency and optimal convergence.

Contribution

It introduces a novel robust support estimation approach for heteroscedastic, heavy-tailed errors with theoretical guarantees and practical efficiency.

Findings

Sign consistency and optimal convergence rates established.

Method performs well in simulations with heavy-tailed errors.

Supports heteroscedastic, heavy-tailed data in high-dimensional settings.

Abstract

A current strand of research in high-dimensional statistics deals with robustifying the available methodology with respect to deviations from the pervasive light-tail assumptions. In this paper we consider a linear mean regression model with random design and potentially heteroscedastic, heavy-tailed errors, and investigate support estimation in this framework. We use a strictly convex, smooth variant of the Huber loss function with tuning parameter depending on the parameters of the problem, as well as the adaptive LASSO penalty for computational efficiency. For the resulting estimator we show sign-consistency and optimal rates of convergence in the $\ell_\infty$ norm as in the homoscedastic, light-tailed setting. In our analysis, we have to deal with the issue that the support of the target parameter in the linear mean regression model and its robustified version may differ substantially even for small values of the tuning parameter of the Huber loss function. Simulations illustrate the favorable numerical performance of the proposed methodology.