A Dimension Reduction Technique for Large-scale Structured Sparse Optimization Problems with Application to Convex Clustering

TL;DR

This paper introduces adaptive sieving techniques that significantly speed up large-scale structured sparse convex optimization, especially in convex clustering, by accelerating existing algorithms like SSNAL and ADMM.

Contribution

The paper presents novel adaptive sieving methods that are solver independent and proven to converge, enhancing the efficiency of large-scale convex optimization with structured sparsity.

Findings

Accelerates SSNAL by over 7 times

Speeds up ADMM by more than 14 times

Provides finite convergence guarantees for the proposed techniques

Abstract

In this paper, we propose a novel adaptive sieving (AS) technique and an enhanced AS (EAS) technique, which are solver independent and could accelerate optimization algorithms for solving large scale convex optimization problems with intrinsic structured sparsity. We establish the finite convergence property of the AS technique and the EAS technique with inexact solutions of the reduced subproblems. As an important application, we apply the AS technique and the EAS technique on the convex clustering model, which could accelerate the state-of-the-art algorithm SSNAL by more than 7 times and the algorithm ADMM by more than 14 times.