Small Tuning Parameter Selection for the Debiased Lasso

TL;DR

This paper explores how smaller tuning parameters in the debiased Lasso can reduce bias without increasing variance, providing a data-driven method for optimal selection and demonstrating its effectiveness through simulations and real data.

Contribution

It introduces a new tuning parameter choice for the debiased Lasso that improves bias-variance trade-off and proposes a consistent data-driven selection procedure.

Findings

Bias can be reduced with tuning parameter of order 1/√n.

Asymptotic normality holds under s₀ = o(√(n/ log p)).

Proposed method yields confidence intervals with good coverage.

Abstract

In this study, we investigate the bias and variance properties of the debiased Lasso in linear regression when the tuning parameter of the node-wise Lasso is selected to be smaller than in previous studies. We consider the case where the number of covariates $p$ is bounded by a constant multiple of the sample size $n$. First, we show that the bias of the debiased Lasso can be reduced without diverging the asymptotic variance by setting the order of the tuning parameter to $1/\sqrt{n}$.This implies that the debiased Lasso has asymptotic normality provided that the number of nonzero coefficients $s_0$ satisfies $s_0=o(\sqrt{n/\log p})$, whereas previous studies require $s_0 =o(\sqrt{n}/\log p)$ if no sparsity assumption is imposed on the precision matrix. Second, we propose a data-driven tuning parameter selection procedure for the node-wise Lasso that is consistent with our theoretical results. Simulation studies show that our procedure yields confidence intervals with good coverage properties in various settings. We also present a real economic data example to demonstrate the efficacy of our selection procedure.