Epinet for Content Cold Start

TL;DR

This paper applies epinet-based uncertainty estimation to online recommendation systems, improving user engagement and traffic efficiency by balancing exploration and exploitation in content cold start scenarios.

Contribution

It is the first to demonstrate the use of epinets for uncertainty quantification in real-world recommendation systems, specifically on Facebook Reels.

Findings

Improved user engagement on Facebook Reels.

Enhanced traffic efficiency through better exploration.

Successful scaling of epinet methods to complex neural network models.

Abstract

The exploding popularity of online content and its user base poses an evermore challenging matching problem for modern recommendation systems. Unlike other frontiers of machine learning such as natural language, recommendation systems are responsible for collecting their own data. Simply exploiting current knowledge can lead to pernicious feedback loops but naive exploration can detract from user experience and lead to reduced engagement. This exploration-exploitation trade-off is exemplified in the classic multi-armed bandit problem for which algorithms such as upper confidence bounds (UCB) and Thompson sampling (TS) demonstrate effective performance. However, there have been many challenges to scaling these approaches to settings which do not exhibit a conjugate prior structure. Recent scalable approaches to uncertainty quantification via epinets have enabled efficient approximations of Thompson sampling even when the learning model is a complex neural network. In this paper, we demonstrate the first application of epinets to an online recommendation system. Our experiments demonstrate improvements in both user traffic and engagement efficiency on the Facebook Reels online video platform.