Method-of-Moments Inference for GLMs and Doubly Robust Functionals under Proportional Asymptotics

TL;DR

This paper introduces a method-of-moments approach for consistent, asymptotically normal inference of regression coefficients and SNR in high-dimensional GLMs, applicable under proportional asymptotics and Gaussian covariates.

Contribution

It develops a novel estimation technique that avoids high-dimensional nuisance estimation and hyperparameter tuning, extending to non-Gaussian covariates under certain conditions.

Findings

Derives consistent and asymptotically normal estimators for high-dimensional GLMs.

Shows universality of results beyond Gaussian covariates.

Demonstrates practical effectiveness through numerical experiments.

Abstract

In this paper, we consider the estimation of regression coefficients and signal-to-noise (SNR) ratio in high-dimensional Generalized Linear Models (GLMs), and explore their implications in inferring popular estimands such as average treatment effects in high-dimensional observational studies. Under the ``proportional asymptotic'' regime and Gaussian covariates with known (population) covariance $\Sigma$, we derive Consistent and Asymptotically Normal (CAN) estimators of our targets of inference through a Method-of-Moments type of estimators that bypasses estimation of high dimensional nuisance functions and hyperparameter tuning altogether. Additionally, under non-Gaussian covariates, we demonstrate universality of our results under certain additional assumptions on the regression coefficients and $\Sigma$. We also demonstrate that knowing $\Sigma$ is not essential to our proposed methodology when the sample covariance matrix estimator is invertible. Finally, we complement our theoretical results with numerical experiments and comparisons with existing literature.