Perturbed Double Machine Learning: Nonstandard Inference Beyond the Parametric Length

TL;DR

This paper introduces Perturbed Double Machine Learning, a method that provides valid inference for low-dimensional parameters even when nuisance estimators converge slowly, by injecting randomness and filtering perturbations.

Contribution

It proposes a novel perturbation-based approach that relaxes convergence rate requirements for nuisance estimators in DML, ensuring valid inference in broader settings.

Findings

Valid coverage achieved with slower nuisance convergence rates.

Method extends to various machine learning nuisance learners.

Simulations demonstrate improved coverage over existing methods.

Abstract

We study inference on a low-dimensional functional $\beta$ in the presence of infinite-dimensional nuisance parameters. Classical inferential methods are typically based on Wald intervals, whose large-sample validity rests on asymptotic negligibility of nuisance error; for example, influence-curve based estimators (Double/Debiased Machine Learning, DML) are asymptotically Gaussian when nuisance estimators converge faster than $n^{-1/4}$. Although such negligibility can hold even in nonparametric classes, it can be restrictive. To relax this requirement, we propose Perturbed Double Machine Learning, which ensures valid inference even when nuisance estimators converge slower than $n^{-1/4}$. Our proposal is to (i) inject randomness into the nuisance estimation step to generate perturbed nuisance models, each yielding an estimate of $\beta$ and a Wald interval, and (ii) filter out perturbations whose deviations from the original DML estimate exceed a threshold. For Lasso nuisance learners, we show that, with high probability, at least one perturbation yields nuisance estimates sufficiently close to the truth, so the associated estimator of $\beta$ is close to an oracle with known nuisances. The union of retained intervals delivers valid coverage even when the DML estimator converges slower than $n^{-1/2}$. The framework extends to general machine-learning nuisance learners, and simulations show coverage when state-of-the-art methods fail.