Item Level Exploration Traffic Allocation in Large-scale Recommendation Systems

TL;DR

This paper presents a novel exploration system for large-scale recommender systems that efficiently allocates traffic to new items, improving their discoverability and addressing the cold start problem through a learned probabilistic model and adaptive traffic distribution.

Contribution

It introduces a scalable, adaptive exploration approach using a learned discoverability model to enhance new item visibility in large-scale recommendation platforms.

Findings

Significant increase in new item discoverability

Efficient cold-start process demonstrated in production

Improved long-term recommendation quality

Abstract

This paper contributes to addressing the item cold start problem in large-scale recommender systems, focusing on how to efficiently gain initial visibility for newly ingested content. We propose an exploration system designed to efficiently allocate impressions to these fresh items. Our approach leverages a learned probabilistic model to predict an item's discoverability, which then informs a scalable and adaptive traffic allocation strategy. This system intelligently distributes exploration budgets, optimizing for the long-term benefit of the recommendation platform. The impact is a demonstrably more efficient cold-start process, leading to a significant increase in the discoverability of new content and ultimately enriching the item corpus available for exploitation, as evidenced by its successful deployment in a large-scale production environment.