Task-specific experimental design for treatment effect estimation

TL;DR

This paper introduces a task-specific experimental design method for causal effect estimation that outperforms traditional RCTs in efficiency by tailoring sampling strategies to specific downstream applications.

Contribution

The authors develop a novel, adaptable experimental design approach that customizes sampling strategies for different tasks, improving data efficiency in causal inference.

Findings

Outperforms benchmarks across various tasks and datasets.

Requires significantly less data than RCTs to achieve similar performance.

Effective in real-world targeted marketing scenarios.

Abstract

Understanding causality should be a core requirement of any attempt to build real impact through AI. Due to the inherent unobservability of counterfactuals, large randomised trials (RCTs) are the standard for causal inference. But large experiments are generically expensive, and randomisation carries its own costs, e.g. when suboptimal decisions are trialed. Recent work has proposed more sample-efficient alternatives to RCTs, but these are not adaptable to the downstream application for which the causal effect is sought. In this work, we develop a task-specific approach to experimental design and derive sampling strategies customised to particular downstream applications. Across a range of important tasks, real-world datasets, and sample sizes, our method outperforms other benchmarks, e.g. requiring an order-of-magnitude less data to match RCT performance on targeted marketing tasks.