GAP Safe Screening Rules for Sparse-Group-Lasso

TL;DR

This paper introduces new safe screening rules for Sparse-Group Lasso that significantly accelerate the optimization process by discarding irrelevant features and groups early, especially in high-dimensional linear regression tasks.

Contribution

It adapts recent safe screening rules to Sparse-Group Lasso, enabling faster convergence and computational efficiency in high-dimensional sparse modeling.

Findings

Significant reduction in computation time with the new screening rules.

Effective discarding of irrelevant features and groups during optimization.

Applicable to coordinate descent methods for Sparse-Group Lasso.

Abstract

In high dimensional settings, sparse structures are crucial for efficiency, either in term of memory, computation or performance. In some contexts, it is natural to handle more refined structures than pure sparsity, such as for instance group sparsity. Sparse-Group Lasso has recently been introduced in the context of linear regression to enforce sparsity both at the feature level and at the group level. We adapt to the case of Sparse-Group Lasso recent safe screening rules that discard early in the solver irrelevant features/groups. Such rules have led to important speed-ups for a wide range of iterative methods. Thanks to dual gap computations, we provide new safe screening rules for Sparse-Group Lasso and show significant gains in term of computing time for a coordinate descent implementation.