Differentiable Unbiased Online Learning to Rank

TL;DR

This paper introduces PDGD, a novel differentiable online learning to rank method that effectively optimizes non-linear models like neural networks, outperforming existing methods in speed and accuracy while maintaining unbiased gradients.

Contribution

The paper presents PDGD, a new unbiased, differentiable OLTR approach that extends effective optimization to non-linear models such as neural networks.

Findings

PDGD outperforms existing OLTR methods in learning speed and convergence.

Using neural networks with PDGD yields better performance than linear models.

PDGD maintains unbiased gradients and improves user experience.

Abstract

Online Learning to Rank (OLTR) methods optimize rankers based on user interactions. State-of-the-art OLTR methods are built specifically for linear models. Their approaches do not extend well to non-linear models such as neural networks. We introduce an entirely novel approach to OLTR that constructs a weighted differentiable pairwise loss after each interaction: Pairwise Differentiable Gradient Descent (PDGD). PDGD breaks away from the traditional approach that relies on interleaving or multileaving and extensive sampling of models to estimate gradients. Instead, its gradient is based on inferring preferences between document pairs from user clicks and can optimize any differentiable model. We prove that the gradient of PDGD is unbiased w.r.t. user document pair preferences. Our experiments on the largest publicly available Learning to Rank (LTR) datasets show considerable and significant improvements under all levels of interaction noise. PDGD outperforms existing OLTR methods both in terms of learning speed as well as final convergence. Furthermore, unlike previous OLTR methods, PDGD also allows for non-linear models to be optimized effectively. Our results show that using a neural network leads to even better performance at convergence than a linear model. In summary, PDGD is an efficient and unbiased OLTR approach that provides a better user experience than previously possible.