TEA-Time: Transporting Effects Across Time

TL;DR

This paper introduces methods to transport treatment effects across time in randomized trials, formalizing the estimand, proposing identification strategies, and evaluating their performance with real and simulated data.

Contribution

It formalizes the transported average treatment effect across time, proposes two identification strategies, and develops efficient estimators for practical application.

Findings

Common arm strategy is more precise but can be biased under certain temporal dependencies.

Replicated trials strategy better tracks ground truth when temporal effects depend on intervention-measurement gap.

Simulation studies reveal conditions under which each strategy is reliable or fails.

Abstract

Treatment effects estimated from a randomized controlled trial are local not only to the study population but also to the time at which the trial was conducted. The literature on generalizing experimental findings to new populations is extensive, yet transporting effects across time has received far less attention, and even defining the target estimand is nonobvious. We formalize the transported average treatment effect under a separable temporal effects assumption, derive two identification strategies: replicated trials and common arm, and develop doubly robust, semiparametrically efficient estimators for each. Applied to a large archive of headline A/B tests, the common arm strategy is substantially more precise but exhibits systematic bias when the temporal factor depends on the gap between intervention and measurement rather than on measurement time alone, while the replicated trials strategy, which allows this dependence, tracks the ground truth more faithfully. Simulation studies investigate when each strategy is reliable and when it silently fails.