High Dimensional Gaussian and Bootstrap Approximations in Generalized Linear Models

TL;DR

This paper studies Gaussian and Bootstrap approximations for high-dimensional generalized linear models, establishing conditions under which these methods are valid, especially considering sparsity and different growth regimes of the dimension.

Contribution

It provides new theoretical results on the validity of Gaussian and Bootstrap approximations in high-dimensional GLMs, including cases with sparsity and exponential dimension growth.

Findings

Gaussian approximation holds for certain dimension growth rates in regime (I)

Bootstrap methods are valid over convex sets and Euclidean balls under specified conditions

Bootstrap approximations remain valid in sparse regimes with high-dimensional Lasso estimators

Abstract

Generalized Linear Model (or GLM) extends the ordinary linear regression by linking the mean of the response variable to covariates through appropriate link functions. GLM is widely used in the analysis of datasets arising from diverse fields including medical sciences, clinical trials, population surveys and risk analysis. In this paper, we investigate the Gaussian and Bootstrap approximations of GLM under two separate high dimensional regimes: (I) when the dimension $d$ grows slower than $n$ and (II) when $d$ grows exponentially with $n$. Under regime (I), we essentially show that the Gaussian approximation holds over the collection of Borel convex sets when $d = o\big(n^{2/5}\big)$ and over the collection of Euclidean balls when $d = o\big(n^{1/2}\big)$. We further devise two high dimensional Bootstrap methods which are valid over the collections of Borel convex sets and Euclidean balls under the same dimension growth rates. Then we move to regime (II) where we invoke sparsity to GLM through Lasso. We show that the high dimensional Gaussian approximation fails under regime (II). However, the Bootstrap approximations over convex sets and Euclidean balls are valid for the relevant part of the GLM estimator provided $\log d = o\big(n^{2\tau/3}\big)$ and the number of non-zero regression parameters is $o\big(n^{1/3- 4\tau/3}\big)$, when the Lasso penalty $\lambda_n \sim n^{1/2 + \tau}$, for some $\tau \in (0, 1/4)$. Simulation studies confirm the strong finite-sample performance of our proposed Bootstrap methods under both regime (I) and (II). We also implement our methods on real datasets.