An error bound for Lasso and Group Lasso in high dimensions

TL;DR

This paper derives new high-dimensional estimation bounds for Lasso and Group Lasso, showing their optimality and conditions under which Group Lasso outperforms Lasso.

Contribution

It provides the first minimax optimal bounds for Lasso and improved bounds for Group Lasso in high dimensions, with insights on their comparative performance.

Findings

Lasso bounds match the minimax rate $(k^*/n) \log(p/k^*)$

Group Lasso bounds improve over previous results

Group Lasso outperforms Lasso when the signal is strongly group-sparse

Abstract

We leverage recent advances in high-dimensional statistics to derive new L2 estimation upper bounds for Lasso and Group Lasso in high-dimensions. For Lasso, our bounds scale as $(k^*/n) \log(p/k^*)$---$n\times p$ is the size of the design matrix and $k^*$ the dimension of the ground truth $\boldsymbol{\beta}^*$---and match the optimal minimax rate. For Group Lasso, our bounds scale as $(s^*/n) \log\left( G / s^* \right) + m^* / n$---$G$ is the total number of groups and $m^*$ the number of coefficients in the $s^*$ groups which contain $\boldsymbol{\beta}^*$---and improve over existing results. We additionally show that when the signal is strongly group-sparse, Group Lasso is superior to Lasso.