Asymptotic Statistical Analysis of Sparse Group LASSO via Approximate Message Passing Algorithm

TL;DR

This paper provides an asymptotic analysis of Sparse Group LASSO using approximate message passing, revealing how group information and penalty proportions influence its statistical performance in high-dimensional regression.

Contribution

It introduces an AMP-based method for analyzing SGL, deriving exact asymptotic characterizations and demonstrating the benefits of group information in sparse recovery.

Findings

SGL with small gamma benefits from group information

AMP provides precise asymptotic analysis of SGL

SGL outperforms other models in recovery and error metrics

Abstract

Sparse Group LASSO (SGL) is a regularized model for high-dimensional linear regression problems with grouped covariates. SGL applies $l_1$ and $l_2$ penalties on the individual predictors and group predictors, respectively, to guarantee sparse effects both on the inter-group and within-group levels. In this paper, we apply the approximate message passing (AMP) algorithm to efficiently solve the SGL problem under Gaussian random designs. We further use the recently developed state evolution analysis of AMP to derive an asymptotically exact characterization of SGL solution. This allows us to conduct multiple fine-grained statistical analyses of SGL, through which we investigate the effects of the group information and $\gamma$ (proportion of $\ell_1$ penalty). With the lens of various performance measures, we show that SGL with small $\gamma$ benefits significantly from the group information and can outperform other SGL (including LASSO) or regularized models which do not exploit the group information, in terms of the recovery rate of signal, false discovery rate and mean squared error.