Safe Screening Rules for Generalized Double Sparsity Learning

TL;DR

This paper introduces safe screening rules for generalized double sparsity learning, enabling early elimination of inactive features/groups to improve computational efficiency in high-dimensional sparse modeling.

Contribution

It proposes a novel framework with safe screening rules for simultaneous feature and group sparsity, enhancing efficiency in high-dimensional regularized learning.

Findings

Developed fast safe screening rules for sparsity regularization

The screening rules guarantee to discard inactive features/groups

Significant computational speed-up achieved in experiments

Abstract

In a high-dimensional setting, sparse model has shown its power in computational and statistical efficiency. We consider variables selection problem with a broad class of simultaneous sparsity regularization, enforcing both feature-wise and group-wise sparsity at the same time. The analysis leverages an introduction of $\epsilon q$-norm in vector space, which is proved to has close connection with the mixture regularization and naturally leads to a dual formulation. Properties of primal/dual optimal solution and optimal values are discussed, which motivates the design of screening rules. We several fast safe screening rules in the general framework, rules that discard inactive features/groups at an early stage that are guaranteed to be inactive in the exact solution, leading to a significant gain in computation speed.