Lass-0: sparse non-convex regression by local search

TL;DR

Lass-0 is a novel method that refines L1-regularized solutions to find sparser, more accurate models by employing a local search for L0 regularization, improving support recovery and parsimony.

Contribution

The paper introduces Lass-0, an efficient local search algorithm that improves upon L1 regularization for sparse linear regression by approximating L0 solutions.

Findings

Lass-0 solutions better recover true sparse support in synthetic data.

Lass-0 finds more parsimonious models with similar accuracy on real data.

Theoretical consistency results under orthogonality conditions.

Abstract

We compute approximate solutions to L0 regularized linear regression using L1 regularization, also known as the Lasso, as an initialization step. Our algorithm, the Lass-0 ("Lass-zero"), uses a computationally efficient stepwise search to determine a locally optimal L0 solution given any L1 regularization solution. We present theoretical results of consistency under orthogonality and appropriate handling of redundant features. Empirically, we use synthetic data to demonstrate that Lass-0 solutions are closer to the true sparse support than L1 regularization models. Additionally, in real-world data Lass-0 finds more parsimonious solutions than L1 regularization while maintaining similar predictive accuracy.