Uplifting Bandits

TL;DR

This paper introduces a multi-armed bandit model for uplift modeling in marketing and recommender systems, proposing UCB algorithms with regret bounds and demonstrating their effectiveness on synthetic and real data.

Contribution

It develops a novel multi-armed bandit framework for uplift modeling, including algorithms and theoretical analysis for unknown baselines and variables.

Findings

UCB-style algorithms achieve sublinear regret bounds.

Estimating uplifts improves policy performance.

Algorithms perform well on synthetic and real datasets.

Abstract

We introduce a multi-armed bandit model where the reward is a sum of multiple random variables, and each action only alters the distributions of some of them. After each action, the agent observes the realizations of all the variables. This model is motivated by marketing campaigns and recommender systems, where the variables represent outcomes on individual customers, such as clicks. We propose UCB-style algorithms that estimate the uplifts of the actions over a baseline. We study multiple variants of the problem, including when the baseline and affected variables are unknown, and prove sublinear regret bounds for all of these. We also provide lower bounds that justify the necessity of our modeling assumptions. Experiments on synthetic and real-world datasets show the benefit of methods that estimate the uplifts over policies that do not use this structure.