Efficient Heterogeneous Treatment Effect Estimation With Multiple Experiments and Multiple Outcomes

TL;DR

This paper introduces a scalable tensor factorization approach called LR-learner for estimating heterogeneous treatment effects across multiple experiments and outcomes, significantly improving precision by leveraging correlations.

Contribution

It proposes a novel low-rank tensor factorization framework for joint HTE estimation across multiple experiments and outcomes, enhancing accuracy over traditional methods.

Findings

LR-learner outperforms independent estimation in synthetic data

Method achieves higher precision in real-world A/B test data

Scalable approach suitable for large-scale experimental settings

Abstract

Learning heterogeneous treatment effects (HTEs) is an important problem across many fields. Most existing methods consider the setting with a single treatment arm and a single outcome metric. However, in many real world domains, experiments are run consistently - for example, in internet companies, A/B tests are run every day to measure the impacts of potential changes across many different metrics of interest. We show that even if an analyst cares only about the HTEs in one experiment for one metric, precision can be improved greatly by analyzing all of the data together to take advantage of cross-experiment and cross-outcome metric correlations. We formalize this idea in a tensor factorization framework and propose a simple and scalable model which we refer to as the low rank or LR-learner. Experiments in both synthetic and real data suggest that the LR-learner can be much more precise than independent HTE estimation.