Experimenting, Fast and Slow: Bayesian Optimization of Long-term Outcomes with Online Experiments

TL;DR

This paper introduces a Bayesian optimization method that efficiently combines quick, biased experiments and offline proxies with long-term online experiments to optimize system performance over extended periods.

Contribution

It presents a novel approach that accelerates long-term outcome optimization by integrating fast experiments and offline proxies with traditional online experiments.

Findings

Effective in reducing experiment duration

Improves accuracy of long-term outcome estimation

Enables optimization over large action spaces

Abstract

Online experiments in internet systems, also known as A/B tests, are used for a wide range of system tuning problems, such as optimizing recommender system ranking policies and learning adaptive streaming controllers. Decision-makers generally wish to optimize for long-term treatment effects of the system changes, which often requires running experiments for a long time as short-term measurements can be misleading due to non-stationarity in treatment effects over time. The sequential experimentation strategies--which typically involve several iterations--can be prohibitively long in such cases. We describe a novel approach that combines fast experiments (e.g., biased experiments run only for a few hours or days) and/or offline proxies (e.g., off-policy evaluation) with long-running, slow experiments to perform sequential, Bayesian optimization over large action spaces in a short amount of time.