Dual Feature Reduction for the Sparse-group Lasso and its Adaptive Variant

TL;DR

This paper introduces Dual Feature Reduction (DFR), a method that significantly decreases computational costs for sparse-group lasso models by applying dual norm screening, without compromising solution accuracy.

Contribution

The paper proposes a novel feature reduction technique, DFR, that enhances the efficiency of sparse-group lasso and its adaptive variant through strong screening rules.

Findings

DFR drastically reduces computational time in high-dimensional data scenarios.

DFR maintains solution optimality despite input space reduction.

Synthetic and real data experiments validate DFR's effectiveness.

Abstract

The sparse-group lasso performs both variable and group selection, simultaneously using the strengths of the lasso and group lasso. It has found widespread use in genetics, a field that regularly involves the analysis of high-dimensional data, due to its sparse-group penalty, which allows it to utilize grouping information. However, the sparse-group lasso can be computationally expensive, due to the added shrinkage complexity, and its additional hyperparameter that needs tuning. This paper presents a novel feature reduction method, Dual Feature Reduction (DFR), that uses strong screening rules for the sparse-group lasso and the adaptive sparse-group lasso to reduce their input space before optimization, without affecting solution optimality. DFR applies two layers of screening through the application of dual norms and subdifferentials. Through synthetic and real data studies, it is shown that DFR drastically reduces the computational cost under many different scenarios.