Principal component-guided sparse regression

TL;DR

The paper introduces pcLasso, a supervised learning method that combines lasso sparsity with principal component guidance, especially effective for high-dimensional data with grouped features, improving feature and group selection.

Contribution

It proposes a novel method, pcLasso, integrating principal component information into sparse regression, with theoretical support and demonstrated advantages on simulated and real datasets.

Findings

pcLasso outperforms traditional lasso in high-dimensional settings.

The method effectively selects relevant feature groups.

Theoretical analysis supports the method's consistency.

Abstract

We propose a new method for supervised learning, especially suited to wide data where the number of features is much greater than the number of observations. The method combines the lasso ($\ell_1$) sparsity penalty with a quadratic penalty that shrinks the coefficient vector toward the leading principal components of the feature matrix. We call the proposed method the "principal components lasso" ("pcLasso"). The method can be especially powerful if the features are pre-assigned to groups (such as cell-pathways, assays or protein interaction networks). In that case, pcLasso shrinks each group-wise component of the solution toward the leading principal components of that group. In the process, it also carries out selection of the feature groups. We provide some theory for this method and illustrate it on a number of simulated and real data examples.