Autotune: fast, accurate, and automatic tuning parameter selection for Lasso

TL;DR

Autotune is an automatic, efficient method for tuning Lasso parameters that improves model selection and generalization, especially in low signal-to-noise scenarios, with applications demonstrated on financial data.

Contribution

The paper introduces autotune, a novel automatic tuning strategy for Lasso that optimizes Gaussian log-likelihood, providing better performance and new tools for high-dimensional inference.

Findings

Autotune outperforms existing methods in speed and accuracy.

It offers a new estimator for noise standard deviation.

It includes a visual diagnostic for sparsity assumptions.

Abstract

Least absolute shrinkage and selection operator (Lasso), a popular method for high-dimensional regression, is now used widely for estimating high-dimensional time series models such as the vector autoregression (VAR). Selecting its tuning parameter efficiently and accurately remains a challenge, despite the abundance of available methods for doing so. We propose $\mathsf{autotune}$, a strategy for Lasso to automatically tune itself by optimizing a penalized Gaussian log-likelihood alternately over regression coefficients and noise standard deviation. Using extensive simulation experiments on regression and VAR models, we show that $\mathsf{autotune}$ is faster, and provides better generalization and model selection than established alternatives in low signal-to-noise regimes. In the process, $\mathsf{autotune}$ provides a new estimator of noise standard deviation that can be used for high-dimensional inference, and a new visual diagnostic procedure for checking the sparsity assumption on regression coefficients. Finally, we demonstrate the utility of $\mathsf{autotune}$ on a real-world financial data set. An R package based on C++ is also made publicly available on Github.