Sparse Group Lasso: Optimal Sample Complexity, Convergence Rate, and Statistical Inference

TL;DR

This paper analyzes the sparse group Lasso in high-dimensional linear regression, establishing optimal sample complexity, convergence rates, and statistical inference methods for both noiseless and noisy cases.

Contribution

It provides matching upper and lower bounds for sample complexity and estimation error, and investigates the asymptotic properties of the debiased sparse group Lasso.

Findings

Matching bounds for sample complexity in noiseless case

Minimax bounds for estimation error in noisy case

Asymptotic properties of debiased sparse group Lasso

Abstract

We study sparse group Lasso for high-dimensional double sparse linear regression, where the parameter of interest is simultaneously element-wise and group-wise sparse. This problem is an important instance of the simultaneously structured model -- an actively studied topic in statistics and machine learning. In the noiseless case, matching upper and lower bounds on sample complexity are established for the exact recovery of sparse vectors and for stable estimation of approximately sparse vectors, respectively. In the noisy case, upper and matching minimax lower bounds for estimation error are obtained. We also consider the debiased sparse group Lasso and investigate its asymptotic property for the purpose of statistical inference. Finally, numerical studies are provided to support the theoretical results.