Simultaneous Inference for High-dimensional Linear Models

TL;DR

This paper introduces a bootstrap-assisted method for simultaneous inference in high-dimensional sparse linear models, capable of handling exponentially larger dimensions than sample size, with theoretical guarantees and extensions to generalized linear models.

Contribution

It develops a novel simultaneous testing procedure based on de-sparsifying Lasso, with improved power, error control, and applicability to generalized linear models.

Findings

Method achieves asymptotic significance level control

Demonstrates strong power even with non-Gaussian errors

Extends to generalized linear models with convex loss functions

Abstract

This paper proposes a bootstrap-assisted procedure to conduct simultaneous inference for high dimensional sparse linear models based on the recent de-sparsifying Lasso estimator (van de Geer et al. 2014). Our procedure allows the dimension of the parameter vector of interest to be exponentially larger than sample size, and it automatically accounts for the dependence within the de-sparsifying Lasso estimator. Moreover, our simultaneous testing method can be naturally coupled with the margin screening (Fan and Lv 2008) to enhance its power in sparse testing with a reduced computational cost, or with the step-down method (Romano and Wolf 2005) to provide a strong control for the family-wise error rate. In theory, we prove that our simultaneous testing procedure asymptotically achieves the pre-specified significance level, and enjoys certain optimality in terms of its power even when the model errors are non-Gaussian. Our general theory is also useful in studying the support recovery problem. To broaden the applicability, we further extend our main results to generalized linear models with convex loss functions. The effectiveness of our methods is demonstrated via simulation studies.