Within Group Variable Selection through the Exclusive Lasso

TL;DR

This paper introduces the Exclusive Lasso, a convex regularization method for selecting at least one variable from each group in predictive models, with scalable algorithms and demonstrated effectiveness on simulated and real NMR data.

Contribution

The paper proposes the Exclusive Lasso, a novel convex penalty for structured variable selection ensuring at least one variable per group, along with scalable algorithms and statistical analysis.

Findings

Effective in simulations for structured variable selection.

Successfully applied to NMR spectroscopy data.

Ensures at least one variable from each group in predictions.

Abstract

Many data sets consist of variables with an inherent group structure. The problem of group selection has been well studied, but in this paper, we seek to do the opposite: our goal is to select at least one variable from each group in the context of predictive regression modeling. This problem is NP-hard, but we study the tightest convex relaxation: a composite penalty that is a combination of the $\ell_1$ and $\ell_2$ norms. Our so-called Exclusive Lasso method performs structured variable selection by ensuring that at least one variable is selected from each group. We study our method's statistical properties and develop computationally scalable algorithms for fitting the Exclusive Lasso. We study the effectiveness of our method via simulations as well as using NMR spectroscopy data. Here, we use the Exclusive Lasso to select the appropriate chemical shift from a dictionary of possible chemical shifts for each molecule in the biological sample.