Estimation of Treatment Effects Under Nonstationarity via the Truncated Policy Gradient Estimator

TL;DR

This paper introduces the Truncated Policy Gradient estimator for accurately estimating treatment effects in nonstationary dynamic systems, reducing bias and variance compared to existing methods.

Contribution

The paper proposes a novel TPG estimator that accounts for nonstationarity by using outcome trajectories, with theoretical guarantees and practical validation.

Findings

TPG estimator reduces bias and variance in nonstationary settings.

Theoretical proof of a central limit theorem for TPG.

Empirical validation on real-world case studies shows improved performance.

Abstract

Randomized experiments (or A/B tests) are widely used to evaluate interventions in dynamic systems such as recommendation platforms, marketplaces, and digital health. In these settings, interventions affect both current and future system states, so estimating the global average treatment effect (GATE) requires accounting for temporal dynamics, which is especially challenging in the presence of nonstationarity; existing approaches suffer from high bias, high variance, or both. In this paper, we address this challenge via the novel Truncated Policy Gradient (TPG) estimator, which replaces instantaneous outcomes with short-horizon outcome trajectories. The estimator admits a policy-gradient interpretation: it is a truncation of the first-order approximation to the GATE, yielding provable reductions in bias and variance in nonstationary Markovian settings. We further establish a central limit theorem for the TPG estimator and develop a consistent variance estimator that remains valid under nonstationarity with single-trajectory data. We validate our theory with two real-world case studies. The results show that a well-calibrated TPG estimator attains low bias and variance in practical nonstationary settings.