Debiasing the Debiased Lasso with Bootstrap

TL;DR

This paper introduces a bootstrap-based modification of the debiased Lasso estimator for high-dimensional linear models, reducing bias and improving confidence interval accuracy especially with many strong signals.

Contribution

It proposes the BS-DB estimator that outperforms the traditional debiased Lasso in bias reduction and inference accuracy under mild conditions.

Findings

BS-DB has smaller bias than debiased Lasso with many strong signals.

Confidence intervals based on BS-DB are asymptotically valid.

Numerical studies show superior performance of BS-DB over existing methods.

Abstract

We consider statistical inference for a single coordinate of regression coefficients in high-dimensional linear models. Recently, the debiased estimators are popularly used for constructing confidence intervals and hypothesis testing in high-dimensional models. However, some representative numerical experiments show that they tend to be biased for large coefficients, especially when the number of large coefficients dominates the number of small coefficients. In this paper, we propose a modified debiased Lasso estimator based on bootstrap. Let us denote the proposed estimator BS-DB for short. We show that, under the irrepresentable condition and other mild technical conditions, the BS-DB has smaller order of bias than the debiased Lasso in existence of a large proportion of strong signals. If the irrepresentable condition does not hold, the BS-DB is guaranteed to perform no worse than the debiased Lasso asymptotically. Confidence intervals based on the BS-DB are proposed and proved to be asymptotically valid under mild conditions. Our study on the inference problems integrates the properties of the Lasso on variable selection and estimation novelly. The superior performance of the BS-DB over the debiased Lasso is demonstrated via extensive numerical studies.