Large-scale Validation of Counterfactual Learning Methods: A Test-Bed

TL;DR

This paper introduces a large-scale, real-world test-bed for evaluating off-policy learning methods in display advertising, demonstrating that recent algorithms can outperform supervised learning on practical data.

Contribution

It provides a standardized test-bed with real-world data for systematic evaluation of off-policy learning algorithms in advertising.

Findings

Recent off-policy methods outperform supervised learning baselines

The test-bed enables systematic validation of off-policy algorithms

Experimental results validate the effectiveness of advanced off-policy techniques

Abstract

The ability to perform effective off-policy learning would revolutionize the process of building better interactive systems, such as search engines and recommendation systems for e-commerce, computational advertising and news. Recent approaches for off-policy evaluation and learning in these settings appear promising. With this paper, we provide real-world data and a standardized test-bed to systematically investigate these algorithms using data from display advertising. In particular, we consider the problem of filling a banner ad with an aggregate of multiple products the user may want to purchase. This paper presents our test-bed, the sanity checks we ran to ensure its validity, and shows results comparing state-of-the-art off-policy learning methods like doubly robust optimization, POEM, and reductions to supervised learning using regression baselines. Our results show experimental evidence that recent off-policy learning methods can improve upon state-of-the-art supervised learning techniques on a large-scale real-world data set.