Estimating the Lasso's Effective Noise

TL;DR

This paper introduces a bootstrap-based, data-driven estimator for the effective noise in the lasso, improving tuning and inference in high-dimensional linear models with finite-sample guarantees.

Contribution

It develops a fully data-driven bootstrap estimator for the effective noise in the lasso, aiding in tuning and inference without extra tuning parameters.

Findings

The estimator accurately captures the quantiles of the effective noise.

It provides finite-sample guarantees for the estimation.

Application to tuning parameter calibration and high-dimensional inference.

Abstract

Much of the theory for the lasso in the linear model $Y = X \beta^* + \varepsilon$ hinges on the quantity $2 \| X^\top \varepsilon \|_{\infty} / n$, which we call the lasso's effective noise. Among other things, the effective noise plays an important role in finite-sample bounds for the lasso, the calibration of the lasso's tuning parameter, and inference on the parameter vector $\beta^*$. In this paper, we develop a bootstrap-based estimator of the quantiles of the effective noise. The estimator is fully data-driven, that is, does not require any additional tuning parameters. We equip our estimator with finite-sample guarantees and apply it to tuning parameter calibration for the lasso and to high-dimensional inference on the parameter vector $\beta^*$.