Adaptive sieving: A dimension reduction technique for sparse optimization problems

TL;DR

This paper introduces an adaptive sieving strategy that efficiently reduces problem size in sparse optimization, enabling faster solutions for large-scale machine learning models with proven theoretical guarantees.

Contribution

The paper presents a novel adaptive sieving approach that explores solution sparsity and provides theoretical guarantees for finite termination in large-scale sparse optimization.

Findings

Effective reduction in problem size for large-scale models

Theoretical guarantees including finite termination

Demonstrated efficiency through extensive numerical experiments

Abstract

In this paper, we propose an adaptive sieving (AS) strategy for solving general sparse machine learning models by effectively exploring the intrinsic sparsity of the solutions, wherein only a sequence of reduced problems with much smaller sizes need to be solved. We further apply the proposed AS strategy to generate solution paths for large-scale sparse optimization problems efficiently. We establish the theoretical guarantees for the proposed AS strategy including its finite termination property. Extensive numerical experiments are presented in this paper to demonstrate the effectiveness and flexibility of the AS strategy to solve large-scale machine learning models.