Uncertainty-Calibrated Recommendations for Low-Active Users

TL;DR

This paper presents a unified framework for recommender systems that uses model uncertainty to improve user engagement and diversity, especially balancing low-active and high-active users.

Contribution

It introduces a production-ready uncertainty calibration method that applies risk-averse and risk-seeking strategies for different user groups in industrial recommender systems.

Findings

Improved retention and satisfaction for low-active users.

Enhanced diversity and category coverage for high-active users.

Validated on a major livestream platform with significant performance gains.

Abstract

A fundamental challenge in recommender systems is balancing reliability for Low-Active Users (LAUs) with diversity for High-Active Users (HAUs). The key to this balance lies in quantifying model uncertainty, which approximates the risk of prediction errors and reveals the limits of the model's current knowledge. On large-scale short-video and livestream platforms, model uncertainty can warn of low-quality recommendations that may lead to disengagement of LAUs and at the same time identify opportunities to diversify content recommendation for HAUs. To leverage this dichotomy, we introduce a unified, production-ready framework that calibrates uncertainty to drive differentiated strategies. Specifically, we implement a model-uncertainty-based risk-averse deboosting policy for LAUs to suppress unreliable recommendations, while employing a risk-seeking Upper Confidence Bound (UCB) strategy for HAUs to encourage exploration. Validated on a major livestream platform, our framework demonstrates significant improvements in retention (active hours) and satisfaction (quality watch time ratio) for LAUs as well as remarkable increases in interest diversity and category coverage for HAUs, proving the value of uncertainty-aware recommendation in industrial settings.