Personalized Policy Learning through Discrete Experimentation: Theory and Empirical Evidence

TL;DR

This paper develops a deep learning framework for personalized continuous policy learning from RCT data with discrete treatments, addressing limitations of traditional discretization methods and demonstrating significant empirical improvements.

Contribution

The paper introduces a theoretically grounded deep learning framework for personalized policy learning from limited RCT treatment levels, with proven asymptotic properties and practical validation.

Findings

Significant improvement in policy value estimation accuracy.

Enhanced ability to identify optimal personalized policies.

Empirical validation shows superior performance over benchmarks.

Abstract

Randomized Controlled Trials (RCTs), or A/B testing, have become the gold standard for optimizing various operational policies on online platforms. However, RCTs on these platforms typically cover a limited number of discrete treatment levels, while the platforms increasingly face complex operational challenges involving optimizing continuous variables, such as pricing and incentive programs. The current industry practice involves discretizing these continuous decision variables into several treatment levels and selecting the optimal discrete treatment level. This approach, however, often leads to suboptimal decisions as it cannot accurately extrapolate performance for untested treatment levels and fails to account for heterogeneity in treatment effects across user characteristics. This study addresses these limitations by developing a theoretically solid and empirically verified framework to learn personalized continuous policies based on high-dimensional user characteristics, using observations from an RCT with only a discrete set of treatment levels. Specifically, we introduce a deep learning for policy targeting (DLPT) framework that includes both personalized policy value estimation and personalized policy learning. We prove that our policy value estimators are asymptotically unbiased and consistent, and the learned policy achieves a root-n-regret bound. We empirically validate our methods in collaboration with a leading social media platform to optimize incentive levels for content creation. Results demonstrate that our DLPT framework significantly outperforms existing benchmarks, achieving substantial improvements in both evaluating the value of policies for each user group and identifying the optimal personalized policy.