Mitigating Exploitation Bias in Learning to Rank with an Uncertainty-aware Empirical Bayes Approach

TL;DR

This paper introduces EBRank, an uncertainty-aware empirical Bayes method that mitigates exploitation bias in learning-to-rank systems by using non-behavior features for prior estimation and updating relevance with observed behaviors.

Contribution

The paper proposes EBRank, a novel Bayesian ranking algorithm that reduces exploitation bias by separating prior relevance estimation from behavior features and incorporating uncertainty-aware exploration.

Findings

EBRank significantly outperforms existing algorithms on public datasets.

It effectively reduces exploitation bias in learning-to-rank models.

The approach is practical and adaptable to real-world ranking systems.

Abstract

Ranking is at the core of many artificial intelligence (AI) applications, including search engines, recommender systems, etc. Modern ranking systems are often constructed with learning-to-rank (LTR) models built from user behavior signals. While previous studies have demonstrated the effectiveness of using user behavior signals (e.g., clicks) as both features and labels of LTR algorithms, we argue that existing LTR algorithms that indiscriminately treat behavior and non-behavior signals in input features could lead to suboptimal performance in practice. Particularly because user behavior signals often have strong correlations with the ranking objective and can only be collected on items that have already been shown to users, directly using behavior signals in LTR could create an exploitation bias that hurts the system performance in the long run. To address the exploitation bias, we propose EBRank, an empirical Bayes-based uncertainty-aware ranking algorithm. Specifically, to overcome exploitation bias brought by behavior features in ranking models, EBRank uses a sole non-behavior feature based prior model to get a prior estimation of relevance. In the dynamic training and serving of ranking systems, EBRank uses the observed user behaviors to update posterior relevance estimation instead of concatenating behaviors as features in ranking models. Besides, EBRank additionally applies an uncertainty-aware exploration strategy to explore actively, collect user behaviors for empirical Bayesian modeling and improve ranking performance. Experiments on three public datasets show that EBRank is effective, practical and significantly outperforms state-of-the-art ranking algorithms.