The Symmetry of a Simple Optimization Problem in Lasso Screening

TL;DR

This paper uncovers a property that simplifies the dual lasso screening problem by reducing its dimensionality, enabling more efficient computation for large datasets.

Contribution

It reveals that the optimization depends only on feature projections onto a subspace, reducing high-dimensional problems to lower dimensions for improved screening efficiency.

Findings

The problem depends only on feature projections onto a subspace.

This property reduces the dimensionality of the optimization problem.

It offers a way to tighten bounds without increasing computational overhead.

Abstract

Recently dictionary screening has been proposed as an effective way to improve the computational efficiency of solving the lasso problem, which is one of the most commonly used method for learning sparse representations. To address today's ever increasing large dataset, effective screening relies on a tight region bound on the solution to the dual lasso. Typical region bounds are in the form of an intersection of a sphere and multiple half spaces. One way to tighten the region bound is using more half spaces, which however, adds to the overhead of solving the high dimensional optimization problem in lasso screening. This paper reveals the interesting property that the optimization problem only depends on the projection of features onto the subspace spanned by the normals of the half spaces. This property converts an optimization problem in high dimension to much lower dimension, and thus sheds light on reducing the computation overhead of lasso screening based on tighter region bounds.