Sparse Group Selection Through Co-Adaptive Penalties

TL;DR

This paper introduces the Co-adaptive Lasso, a fast and effective method for linear regression with structured group sparsity, outperforming existing methods like Group Lasso and Adaptive Lasso in theory and simulations.

Contribution

The paper proposes the Co-adaptive Lasso, a new method that handles group sparsity efficiently and offers theoretical and empirical advantages over existing techniques.

Findings

The Co-adaptive Lasso is computationally fast and simple.

It has theoretical benefits over the traditional Lasso.

Performs competitively in simulations compared to Group Lasso and Adaptive Lasso.

Abstract

Recent work has focused on the problem of conducting linear regression when the number of covariates is very large, potentially greater than the sample size. To facilitate this, one useful tool is to assume that the model can be well approximated by a fit involving only a small number of covariates -- a so called sparsity assumption, which leads to the Lasso and other methods. In many situations, however, the covariates can be considered to be structured, in that the selection of some variables favours the selection of others -- with variables organised into groups entering or leaving the model simultaneously as a special case. This structure creates a different form of sparsity. In this paper, we suggest the Co-adaptive Lasso to fit models accommodating this form of `group sparsity'. The Co-adaptive Lasso is fast and simple to calculate, and we show that it holds theoretical advantages over the Lasso, performs well under a broad set of conclusions, and is very competitive in empirical simulations in comparison with previously suggested algorithms like the Group Lasso and the Adaptive Lasso.