A Unified Framework for Debiased Machine Learning: Riesz Representer Fitting under Bregman Divergence

TL;DR

This paper introduces a unified framework for estimating the Riesz representer using Bregman divergence minimization, enabling automatic covariate balancing and Neyman orthogonalization in debiased machine learning.

Contribution

It proposes generalized Riesz regression that encompasses various divergences, providing a flexible, theoretically grounded approach with convergence guarantees and practical implementation.

Findings

Framework includes divergences like squared distance and KL divergence.

Automatic covariate balancing and Neyman orthogonalization are achieved.

Convergence analyses are provided for RKHS and neural network models.

Abstract

Estimating the Riesz representer is central to debiased machine learning for causal and structural parameter estimation. We propose generalized Riesz regression, a unified framework for estimating the Riesz representer by fitting a representer model via Bregman divergence minimization. This framework includes various divergences as special cases, such as the squared distance and the Kullback--Leibler (KL) divergence, where the former recovers Riesz regression and the latter recovers tailored loss minimization. Under suitable pairs of divergence and model specifications (link functions), the dual problems of the Riesz representer fitting problem correspond to covariate balancing, which we call automatic covariate balancing. Moreover, under the same specifications, the sample average of outcomes weighted by the estimated Riesz representer satisfies Neyman orthogonality even without estimating the regression function, a property we call automatic Neyman orthogonalization. This property not only reduces the estimation error of Neyman orthogonal scores but also clarifies a key distinction between debiased machine learning and targeted maximum likelihood estimation (TMLE). Our framework can also be viewed as a generalization of density ratio fitting under Bregman divergences to Riesz representer estimation, and it applies beyond density ratio estimation. We provide convergence analyses for both reproducing kernel Hilbert space (RKHS) and neural network model classes. A Python package for generalized Riesz regression is released as genriesz and is available at https://github.com/MasaKat0/genriesz.