Making Contextual Decisions with Low Technical Debt

TL;DR

This paper introduces the Decision Service, a comprehensive system for contextual bandit learning that reduces technical debt and improves decision-making in real-world applications through systematic abstractions and reproducibility.

Contribution

The paper presents the first general system for contextual learning that addresses technical debt with novel abstractions, enabling scalable, real-time, and reproducible decision-making.

Findings

Achieved 25-30% click-through improvements in content recommendation.

Realized 18% revenue lift in landing page optimization.

Deployed in applications like tech support and machine failure handling.

Abstract

Applications and systems are constantly faced with decisions that require picking from a set of actions based on contextual information. Reinforcement-based learning algorithms such as contextual bandits can be very effective in these settings, but applying them in practice is fraught with technical debt, and no general system exists that supports them completely. We address this and create the first general system for contextual learning, called the Decision Service. Existing systems often suffer from technical debt that arises from issues like incorrect data collection and weak debuggability, issues we systematically address through our ML methodology and system abstractions. The Decision Service enables all aspects of contextual bandit learning using four system abstractions which connect together in a loop: explore (the decision space), log, learn, and deploy. Notably, our new explore and log abstractions ensure the system produces correct, unbiased data, which our learner uses for online learning and to enable real-time safeguards, all in a fully reproducible manner. The Decision Service has a simple user interface and works with a variety of applications: we present two live production deployments for content recommendation that achieved click-through improvements of 25-30%, another with 18% revenue lift in the landing page, and ongoing applications in tech support and machine failure handling. The service makes real-time decisions and learns continuously and scalably, while significantly lowering technical debt.