Strong rules for discarding predictors in lasso-type problems

TL;DR

This paper introduces strong rules for efficiently discarding predictors in lasso regression, significantly speeding up computations while maintaining high accuracy through simple checks of optimality conditions.

Contribution

It proposes strong rules that are not foolproof but highly effective, and can be combined with KKT checks for safe, efficient predictor screening in convex optimization.

Findings

Strong rules rarely fail in practice.

Significant computational speedups achieved.

Effective in various statistical convex problems.

Abstract

We consider rules for discarding predictors in lasso regression and related problems, for computational efficiency. El Ghaoui et al (2010) propose "SAFE" rules that guarantee that a coefficient will be zero in the solution, based on the inner products of each predictor with the outcome. In this paper we propose strong rules that are not foolproof but rarely fail in practice. These can be complemented with simple checks of the Karush- Kuhn-Tucker (KKT) conditions to provide safe rules that offer substantial speed and space savings in a variety of statistical convex optimization problems.