Data-Driven Off-Policy Estimator Selection: An Application in User Marketing on An Online Content Delivery Service

TL;DR

This paper proposes a data-driven method for selecting the most appropriate off-policy evaluation estimator tailored to specific application settings, demonstrated through real-world online content delivery and marketing scenarios.

Contribution

It introduces a practical estimator selection procedure for off-policy evaluation, addressing the challenge of choosing suitable estimators for different real-world applications.

Findings

Estimator suitability varies with outcome definitions.

The selection procedure effectively identifies appropriate estimators.

Proper estimator selection improves evaluation accuracy in practice.

Abstract

Off-policy evaluation (OPE) is the method that attempts to estimate the performance of decision making policies using historical data generated by different policies without conducting costly online A/B tests. Accurate OPE is essential in domains such as healthcare, marketing or recommender systems to avoid deploying poor performing policies, as such policies may hart human lives or destroy the user experience. Thus, many OPE methods with theoretical backgrounds have been proposed. One emerging challenge with this trend is that a suitable estimator can be different for each application setting. It is often unknown for practitioners which estimator to use for their specific applications and purposes. To find out a suitable estimator among many candidates, we use a data-driven estimator selection procedure for off-policy policy performance estimators as a practical solution. As proof of concept, we use our procedure to select the best estimator to evaluate coupon treatment policies on a real-world online content delivery service. In the experiment, we first observe that a suitable estimator might change with different definitions of the outcome variable, and thus the accurate estimator selection is critical in real-world applications of OPE. Then, we demonstrate that, by utilizing the estimator selection procedure, we can easily find out suitable estimators for each purpose.