Universality of estimators for high-dimensional linear models with block dependency

TL;DR

This paper demonstrates that in high-dimensional linear models with block-dependent covariates, the distribution of estimators remains universal, independent of Gaussianity, under certain moment conditions.

Contribution

It extends the universality property of estimators to models with block-dependent covariates, using a generalized Lindeberg principle and new error bounds.

Findings

Estimators' distributions are universal under block dependence.

Universality holds for various estimators with non-Gaussian covariates.

Developed a generalized Lindeberg principle for block-dependent data.

Abstract

We study the universality property of estimators for high-dimensional linear models, which implies that the distribution of estimators is independent of whether the covariates follow a Gaussian distribution. Recent developments in high-dimensional statistics typically require covariates to strictly follow a Gaussian distribution to precisely characterize the properties of estimators. To relax this Gaussianity requirement, the existing literature has examined conditions under which estimators achieve universality. In particular, independence among the elements of the high-dimensional covariates has played a critical role. In this study, we focus on high-dimensional linear models with covariates exhibiting block dependence, where covariate elements can only be dependent within each block, and show that estimators for such models retain universality. Specifically, we prove that the distribution of estimators with Gaussian covariates can be approximated by the distribution of estimators with non-Gaussian covariates having the same moments under block dependence. To establish this result, we develop a generalized Lindeberg principle suitable for handling block dependencies and derive new error bounds for correlated covariate elements. We further demonstrate the universality result across several different estimators.