Sparse and Robust Linear Regression: An Optimization Algorithm and Its Statistical Properties

TL;DR

This paper introduces an optimization algorithm for sparse linear regression that effectively handles outliers, with proven convergence and support recovery guarantees, enhancing robustness and accuracy in statistical modeling.

Contribution

The paper proposes a novel optimization algorithm for sparse linear regression with outliers, providing theoretical convergence and support recovery guarantees.

Findings

Algorithm converges under certain conditions.

Supports accurate recovery of true coefficient support.

Handles outliers effectively in sparse regression.

Abstract

This paper studies sparse linear regression analysis with outliers in the responses. A parameter vector for modeling outliers is added to the standard linear regression model and then the sparse estimation problem for both coefficients and outliers is considered. The $\ell_{1}$ penalty is imposed for the coefficients, while various penalties including redescending type penalties are for the outliers. To solve the sparse estimation problem, we introduce an optimization algorithm. Under some conditions, we show the algorithmic and statistical convergence property for the coefficients obtained by the algorithm. Moreover, it is shown that the algorithm can recover the true support of the coefficients with probability going to one.