Contextual Dual Learning Algorithm with Listwise Distillation for Unbiased Learning to Rank

TL;DR

This paper introduces a novel dual learning algorithm with listwise distillation for unbiased learning to rank, effectively addressing position and contextual biases using real-world web search data.

Contribution

The paper proposes the Contextual Dual Learning Algorithm with Listwise Distillation (CDLA-LD), a new method that improves unbiased ranking by jointly modeling position and contextual biases with real-world data.

Findings

The proposed method outperforms existing ULTR methods on real-world datasets.

CDLA-LD effectively reduces bias and improves ranking accuracy.

Experimental results validate the approach's robustness and generalization.

Abstract

Unbiased Learning to Rank (ULTR) aims to leverage biased implicit user feedback (e.g., click) to optimize an unbiased ranking model. The effectiveness of the existing ULTR methods has primarily been validated on synthetic datasets. However, their performance on real-world click data remains unclear. Recently, Baidu released a large publicly available dataset of their web search logs. Subsequently, the NTCIR-17 ULTRE-2 task released a subset dataset extracted from it. We conduct experiments on commonly used or effective ULTR methods on this subset to determine whether they maintain their effectiveness. In this paper, we propose a Contextual Dual Learning Algorithm with Listwise Distillation (CDLA-LD) to simultaneously address both position bias and contextual bias. We utilize a listwise-input ranking model to obtain reconstructed feature vectors incorporating local contextual information and employ the Dual Learning Algorithm (DLA) method to jointly train this ranking model and a propensity model to address position bias. As this ranking model learns the interaction information within the documents list of the training set, to enhance the ranking model's generalization ability, we additionally train a pointwise-input ranking model to learn the listwise-input ranking model's capability for relevance judgment in a listwise manner. Extensive experiments and analysis confirm the effectiveness of our approach.