Adaptive Dimension Reduction for Overlapping Group Sparsity

TL;DR

This paper introduces new dual certificates and an adaptive support identification scheme for overlapping group sparsity, significantly accelerating existing optimization algorithms and verified through convergence analysis and experiments.

Contribution

It develops novel dual certificates and an adaptive scheme for overlapping group sparsity, enhancing the efficiency of existing algorithms.

Findings

Accelerates convergence of multiple algorithms

Provides theoretical convergence analysis

Demonstrates practical effectiveness on datasets

Abstract

Typical dimension reduction techniques for nonoverlapping sparse optimization involve screening or sieving strategies based on a dual certificate derived from the first-order optimality condition, approximating the gradients or exploiting certain inherent low-dimensional structure of the sparse solution. In comparison, dimension reduction rules for overlapping group sparsity are generally less developed because the subgradient structure is more complex, making the link between sparsity pattern and the dual variable indirect due to the non-separability. In this work, we propose new dual certificates for overlapping group sparsity and a novel adaptive scheme for identifying the support of the overlapping group LASSO. We demonstrate how this scheme can be integrated into and significantly accelerate existing algorithms, including Primal-Dual splitting method, alternating direction method of multipliers and a recently developed variable projection scheme based on over-parameterization. We provide convergence analysis of the method and verify its practical effectiveness through experiments on standard datasets.