Unifying On- and Off-Policy Variance Reduction Methods

TL;DR

This paper establishes a formal equivalence between online and off-policy variance reduction methods, unifying their statistical frameworks and enhancing understanding for practitioners in experimental design.

Contribution

It proves the mathematical equivalence of key variance reduction techniques across online and off-policy settings, bridging a longstanding methodological divide.

Findings

Online Difference-in-Means equals off-policy Inverse Propensity Scoring with optimal control variate

Regression adjustment methods are structurally equivalent to Doubly Robust estimation

Unified framework guides better application of variance reduction techniques

Abstract

Continuous and efficient experimentation is key to the practical success of user-facing applications on the web, both through online A/B-tests and off-policy evaluation. Despite their shared objective -- estimating the incremental value of a treatment -- these domains often operate in isolation, utilising distinct terminologies and statistical toolkits. This paper bridges that divide by establishing a formal equivalence between their canonical variance reduction methods. We prove that the standard online Difference-in-Means estimator is mathematically identical to an off-policy Inverse Propensity Scoring estimator equipped with an optimal (variance-minimising) additive control variate. Extending this unification, we demonstrate that widespread regression adjustment methods (such as CUPED, CUPAC, and ML-RATE) are structurally equivalent to Doubly Robust estimation. This unified view extends our understanding of commonly used approaches, and can guide practitioners and researchers working on either class of problems.