L1 penalized LAD estimator for high dimensional linear

TL;DR

This paper studies an L1 penalized LAD estimator for high-dimensional linear regression, demonstrating its robustness to various noise distributions and near-oracle performance without needing noise variance knowledge.

Contribution

It introduces a robust L1 penalized LAD method that works under minimal noise assumptions and achieves near-oracle error bounds in high-dimensional settings.

Findings

Achieves near-oracle error bounds with high probability

Works under broad noise distribution assumptions, including Cauchy

Does not require noise variance or moment assumptions

Abstract

In this paper, the high-dimensional sparse linear regression model is considered, where the overall number of variables is larger than the number of observations. We investigate the L1 penalized least absolute deviation method. Different from most of other methods, the L1 penalized LAD method does not need any knowledge of standard deviation of the noises or any moment assumptions of the noises. Our analysis shows that the method achieves near oracle performance, i.e. with large probability, the L2 norm of the estimation error is of order $O(\sqrt{k \log p/n})$. The result is true for a wide range of noise distributions, even for the Cauchy distribution. Numerical results are also presented.