Evaluation and Optimization of Leave-one-out Cross-validation for the Lasso

TL;DR

This paper introduces an efficient algorithm to compute and optimize leave-one-out cross-validation for the lasso, enabling precise hyperparameter tuning and applicability to large datasets.

Contribution

It presents a novel piecewise quadratic algorithm for exact and approximate leave-one-out cross-validation for the lasso, improving hyperparameter selection.

Findings

Algorithm accurately computes leave-one-out CV for the lasso.

It enables optimal hyperparameter selection both globally and locally.

The method is practical for real-world and large datasets.

Abstract

I develop an algorithm to produce the piecewise quadratic that computes leave-one-out cross-validation for the lasso as a function of its hyperparameter. The algorithm can be used to find exact hyperparameters that optimize leave-one-out cross-validation either globally or locally, and its practicality is demonstrated on real-world data sets. I also show how the algorithm can be modified to compute approximate leave-one-out cross-validation, making it suitable for larger data sets.