Semi-parametric efficiency bounds for high-dimensional models

TL;DR

This paper develops a framework to determine semi-parametric efficiency bounds in high-dimensional models, showing that de-sparsified estimators can achieve these bounds and are asymptotically efficient.

Contribution

It introduces a novel approach to derive efficiency bounds in high-dimensional settings and demonstrates that certain de-sparsified estimators attain these bounds.

Findings

De-sparsified estimators achieve the semi-parametric efficiency bounds.

The framework applies to linear regression and Gaussian graphical models.

Asymptotic lower bounds are established using Cramér-Rao and Le Cam's methods.

Abstract

Asymptotic lower bounds for estimation play a fundamental role in assessing the quality of statistical procedures. In this paper we propose a framework for obtaining semi-parametric efficiency bounds for sparse high-dimensional models, where the dimension of the parameter is larger than the sample size. We adopt a semi-parametric point of view: we concentrate on one dimensional functions of a high-dimensional parameter. We follow two different approaches to reach the lower bounds: asymptotic Cram\'er-Rao bounds and Le Cam's type of analysis. Both these approaches allow us to define a class of asymptotically unbiased or "regular" estimators for which a lower bound is derived. Consequently, we show that certain estimators obtained by de-sparsifying (or de-biasing) an $\ell_1$-penalized M-estimator are asymptotically unbiased and achieve the lower bound on the variance: thus in this sense they are asymptotically efficient. The paper discusses in detail the linear regression model and the Gaussian graphical model.