An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

TL;DR

This paper introduces an efficient augmented Lagrangian algorithm utilizing a semismooth Newton method for large-scale sparse group Lasso problems, achieving fast convergence and demonstrating superior computational performance.

Contribution

The paper develops a novel augmented Lagrangian approach with a superlinearly convergent semismooth Newton method tailored for large-scale sparse group Lasso, including explicit Jacobian derivation and second order sparsity exploitation.

Findings

Algorithm converges globally at an arbitrarily fast linear rate.

Explicit generalized Jacobian of the proximal mapping is derived.

Numerical experiments show improved efficiency and robustness.

Abstract

The sparse group Lasso is a widely used statistical model which encourages the sparsity both on a group and within the group level. In this paper, we develop an efficient augmented Lagrangian method for large-scale non-overlapping sparse group Lasso problems with each subproblem being solved by a superlinearly convergent inexact semismooth Newton method. Theoretically, we prove that, if the penalty parameter is chosen sufficiently large, the augmented Lagrangian method converges globally at an arbitrarily fast linear rate for the primal iterative sequence, the dual infeasibility, and the duality gap of the primal and dual objective functions. Computationally, we derive explicitly the generalized Jacobian of the proximal mapping associated with the sparse group Lasso regularizer and exploit fully the underlying second order sparsity through the semismooth Newton method. The efficiency and robustness of our proposed algorithm are demonstrated by numerical experiments on both the synthetic and real data sets.