Automatic debiasing of neural networks via moment-constrained learning

TL;DR

This paper introduces a moment-constrained learning method for neural networks to improve automatic debiasing in estimating causal effects, addressing robustness issues in existing approaches.

Contribution

It proposes a novel moment-constrained learning approach for Riesz representer estimation, enhancing robustness over existing automatic debiasing methods.

Findings

Improved accuracy in causal effect estimation.

Enhanced robustness to hyperparameter tuning.

Outperforms state-of-the-art benchmarks in experiments.

Abstract

Causal and nonparametric estimands in economics and biostatistics can often be viewed as the mean of a linear functional applied to an unknown outcome regression function. Naively learning the regression function and taking a sample mean of the target functional results in biased estimators, and a rich debiasing literature has developed where one additionally learns the so-called Riesz representer (RR) of the target estimand (targeted learning, double ML, automatic debiasing etc.). Learning the RR via its derived functional form can be challenging, e.g. due to extreme inverse probability weights or the need to learn conditional density functions. Such challenges have motivated recent advances in automatic debiasing (AD), where the RR is learned directly via minimization of a bespoke loss. We propose moment-constrained learning as a new RR learning approach that addresses some shortcomings in AD, constraining the predicted moments and improving the robustness of RR estimates to optimization hyperparamters. Though our approach is not tied to a particular class of learner, we illustrate it using neural networks, and evaluate on the problems of average treatment/derivative effect estimation using semi-synthetic data. Our numerical experiments show improved performance versus state of the art benchmarks.