Dynamic Prior Thompson Sampling for Cold-Start Exploration in Recommender Systems

TL;DR

This paper introduces Dynamic Prior Thompson Sampling, a method that improves cold-start exploration in recommender systems by controlling the probability of new items outperforming existing ones, leading to more efficient and predictable exploration.

Contribution

The paper presents a closed-form quadratic solution for setting priors in Thompson Sampling, enabling explicit control over exploration intensity during cold-start in large-scale recommender systems.

Findings

Enhanced exploration control in simulations and online experiments.

Improved efficiency over uniform prior baseline.

Effective management of cold-start bias in recommender systems.

Abstract

Cold-start exploration is a core challenge in large-scale recommender systems: new or data-sparse items must receive traffic to estimate value, but over-exploration harms users and wastes impressions. In practice, Thompson Sampling (TS) is often initialized with a uniform Beta(1,1) prior, implicitly assuming a 50% success rate for unseen items. When true base rates are far lower, this optimistic prior systematically over-allocates to weak items. The impact is amplified by batched policy updates and pipeline latency: for hours, newly launched items can remain effectively "no data," so the prior dominates allocation before feedback is incorporated. We propose Dynamic Prior Thompson Sampling, a prior design that directly controls the probability that a new arm outcompetes the incumbent winner. Our key contribution is a closed-form quadratic solution for the prior mean that enforces P(X_j > Y_k) = epsilon at introduction time, making exploration intensity predictable and tunable while preserving TS Bayesian updates. Across Monte Carlo validation, offline batched simulations, and a large-scale online experiment on a thumbnail personalization system serving millions of users, dynamic priors deliver precise exploration control and improved efficiency versus a uniform-prior baseline.