Non-Plug-In Estimators Could Outperform Plug-In Estimators: a Cautionary Note and a Diagnosis
Hongxiang Qiu

TL;DR
This paper challenges the common belief that plug-in estimators like TMLE outperform non-plug-in estimators such as DML in small samples, showing through simulations that DML can perform better and suggesting diagnostic checks for TMLE stability.
Contribution
It provides a simulation-based comparison of TMLE and DML estimators, highlighting scenarios where TMLE may underperform and proposing fluctuation magnitude as a diagnostic tool.
Findings
DML outperforms some TMLE versions in small samples
TMLE fluctuations are unstable and can indicate poor performance
Checking fluctuation magnitude can diagnose TMLE reliability
Abstract
Objectives: Highly flexible nonparametric estimators have gained popularity in causal inference and epidemiology. Popular examples of such estimators include targeted maximum likelihood estimators (TMLE) and double machine learning (DML). TMLE is often argued or suggested to be better than DML estimators and several other estimators in small to moderate samples -- even if they share the same large-sample properties -- because TMLE is a plug-in estimator and respects the known bounds on the parameter, while other estimators might fall outside the known bounds and yield absurd estimates. However, this argument is not a rigorously proven result and may fail in certain cases. Methods: In a carefully chosen simulation setting, I compare the performance of several versions of TMLE and DML estimators of the average treatment effect among treated in small to moderate samples. Results: In this…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Statistical Process Monitoring
