Ill-Conditioned Orthogonal Scores in Double Machine Learning

TL;DR

This paper investigates how the conditioning of the orthogonal-score Jacobian affects the reliability of double machine learning estimates, especially when residualized treatment variance is small, and proposes diagnostics to assess this fragility.

Contribution

It provides an exact identity for the cross-fitted PLR-DML estimator, derives bounds on nuisance error amplification, and introduces diagnostics for assessing estimator fragility due to ill-conditioning.

Findings

Ill-conditioning can cause confidence interval under-coverage.

The paper offers an exact identity for the estimator without Taylor approximation.

Diagnostics can identify regimes where residual variation weakens inference.

Abstract

Double Machine Learning is often justified by nuisance-rate conditions, yet finite-sample reliability also depends on the conditioning of the orthogonal-score Jacobian. This conditioning is typically assumed rather than tracked. When residualized treatment variance is small, the Jacobian is ill-conditioned and small systematic nuisance errors can be amplified, so nominal confidence intervals may look precise yet systematically under-cover. Our main result is an exact identity for the cross-fitted PLR-DML estimator, with no Taylor approximation. From this identity, we derive a stochastic-order bound that separates oracle noise from a conditioning-amplified nuisance remainder and yields a sufficiency condition for root-n-inference. We further connect the amplification factor to semiparametric efficiency geometry via the Riesz representer and use a triangular-array framework to characterize regimes as residual treatment variation weakens. These results motivate an out-of-fold diagnostic that summarizes the implied amplification scale. We do not propose universal thresholds. Instead, we recommend reporting the diagnostic alongside cross-learner sensitivity summaries as a fragility assessment, illustrated in simulation and an empirical example.