Sharp Structure-Agnostic Lower Bounds for General Linear Functional Estimation

TL;DR

This paper develops a general optimality theory for estimating linear functionals of unknown nuisance parameters, proving the optimality of doubly robust estimators and characterizing minimax rates across different regimes.

Contribution

It introduces a structure-agnostic framework for optimal estimation of linear functionals, establishing the statistical optimality of doubly robust estimators and extending results to various settings.

Findings

Doubly robust estimators are statistically optimal within the framework.

The minimax optimal rate is characterized for general estimation problems.

First-order debiasing achieves optimal error rates in different regimes.

Abstract

We establish a general statistical optimality theory for estimation problems where the target parameter is a linear functional of an unknown nuisance component that must be estimated from data. This formulation covers many causal and predictive parameters and has applications to numerous disciplines. We adopt the structure-agnostic framework introduced by \citet{balakrishnan2023fundamental}, which poses no structural properties on the nuisance functions other than access to black-box estimators that achieve some statistical estimation rate. This framework is particularly appealing when one is only willing to consider estimation strategies that use non-parametric regression and classification oracles as black-box sub-processes. Within this framework, we first prove the statistical optimality of the celebrated and widely used doubly robust estimators for the Average Treatment Effect (ATE), the most central parameter in causal inference. We then characterize the minimax optimal rate under the general formulation. Notably, we differentiate between two regimes in which double robustness can and cannot be achieved and in which first-order debiasing yields different error rates. Our result implies that first-order debiasing is simultaneously optimal in both regimes. We instantiate our theory by deriving optimal error rates that recover existing results and extend to various settings of interest, including the case when the nuisance is defined by generalized regressions and when covariate shift exists for training and test distribution.