Extending Open Bandit Pipeline to Simulate Industry Challenges

TL;DR

This paper extends the Open Bandit Pipeline to simulate real-world industry challenges like delayed rewards and concept drift, aiding practitioners and researchers in developing robust bandit algorithms for e-commerce.

Contribution

It introduces simulation components for industry-specific challenges within the OBP framework, facilitating better testing and development of bandit algorithms.

Findings

Extended OBP with industry challenge simulations

Supports research on delayed reward and concept drift

Aids practitioners in addressing real-world e-commerce issues

Abstract

Bandit algorithms are often used in the e-commerce industry to train Machine Learning (ML) systems when pre-labeled data is unavailable. However, the industry setting poses various challenges that make implementing bandit algorithms in practice non-trivial. In this paper, we elaborate on the challenges of off-policy optimisation, delayed reward, concept drift, reward design, and business rules constraints that practitioners at Booking.com encounter when applying bandit algorithms. Our main contributions is an extension to the Open Bandit Pipeline (OBP) framework. We provide simulation components for some of the above-mentioned challenges to provide future practitioners, researchers, and educators with a resource to address challenges encountered in the e-commerce industry.