Reliable Selection of Heterogeneous Treatment Effect Estimators

TL;DR

This paper introduces a novel, ground-truth-free method for selecting the best heterogeneous treatment effect estimators, ensuring reliable error control and reducing false selections in various benchmark datasets.

Contribution

It proposes a new estimator selection procedure based on a cross-fitted, exponentially weighted test statistic with a two-way sample splitting scheme, providing asymptotic error control without ground-truth effects.

Findings

Reduces false selections compared to existing methods.

Provides reliable error control in heterogeneous treatment effect estimation.

Demonstrates effectiveness across multiple benchmark datasets.

Abstract

We study the problem of selecting the best heterogeneous treatment effect (HTE) estimator from a collection of candidates in settings where the treatment effect is fundamentally unobserved. We cast estimator selection as a multiple testing problem and introduce a ground-truth-free procedure based on a cross-fitted, exponentially weighted test statistic. A key component of our method is a two-way sample splitting scheme that decouples nuisance estimation from weight learning and ensures the stability required for valid inference. Leveraging a stability-based central limit theorem, we establish asymptotic familywise error rate control under mild regularity conditions. Empirically, our procedure provides reliable error control while substantially reducing false selections compared with commonly used methods across ACIC 2016, IHDP, and Twins benchmarks, demonstrating that our method is feasible and powerful even without ground-truth treatment effects.