The Predictive Lasso

TL;DR

The paper introduces the Predictive Lasso, a shrinkage method for variable selection in GLMs that minimizes predictive divergence and maintains predictive accuracy, leveraging existing lasso algorithms.

Contribution

It develops a new lasso-based procedure focused on optimizing predictive performance through Kullback-Leibler divergence minimization in GLMs.

Findings

Achieves similar predictive accuracy to full models with fewer variables

Utilizes existing lasso algorithms for efficient computation

Demonstrates effectiveness through simulations and real data

Abstract

We propose a shrinkage procedure for simultaneous variable selection and estimation in generalized linear models (GLMs) with an explicit predictive motivation. The procedure estimates the coefficients by minimizing the Kullback-Leibler divergence of a set of predictive distributions to the corresponding predictive distributions for the full model, subject to an $l_1$ constraint on the coefficient vector. This results in selection of a parsimonious model with similar predictive performance to the full model. Thanks to its similar form to the original lasso problem for GLMs, our procedure can benefit from available $l_1$-regularization path algorithms. Simulation studies and real-data examples confirm the efficiency of our method in terms of predictive performance on future observations.