Exponential Lasso: robust sparse penalization under heavy-tailed noise and outliers with exponential-type loss

TL;DR

The paper introduces the Exponential Lasso, a robust high-dimensional variable selection method that effectively handles heavy-tailed noise and outliers by using an exponential-type loss function, improving reliability over classical Lasso.

Contribution

It proposes the Exponential Lasso, integrating an exponential loss into the Lasso framework, with theoretical guarantees and an efficient MM algorithm for robust sparse estimation.

Findings

Outperforms classical Lasso in contaminated data scenarios

Achieves strong statistical convergence rates similar to classical Lasso

Maintains robustness against heavy-tailed noise and outliers

Abstract

In high-dimensional statistics, the Lasso is a cornerstone method for simultaneous variable selection and parameter estimation. However, its reliance on the squared loss function renders it highly sensitive to outliers and heavy-tailed noise, potentially leading to unreliable model selection and biased estimates. To address this limitation, we introduce the Exponential Lasso, a novel robust method that integrates an exponential-type loss function within the Lasso framework. This loss function is designed to achieve a smooth trade-off between statistical efficiency under Gaussian noise and robustness against data contamination. Unlike other methods that cap the influence of large residuals, the exponential loss smoothly redescends, effectively downweighting the impact of extreme outliers while preserving near-quadratic behavior for small errors. We establish theoretical guarantees showing that the Exponential Lasso achieves strong statistical convergence rates, matching the classical Lasso under ideal conditions while maintaining its robustness in the presence of heavy-tailed contamination. Computationally, the estimator is optimized efficiently via a Majorization-Minimization (MM) algorithm that iteratively solves a series of weighted Lasso subproblems. Numerical experiments demonstrate that the proposed method is highly competitive, outperforming the classical Lasso in contaminated settings and maintaining strong performance even under Gaussian noise. Our method is implemented in the \texttt{R} package \texttt{heavylasso} available on Github: https://github.com/tienmt/heavylasso