Correcting for Position Bias in Learning to Rank: A Control Function Approach

TL;DR

This paper introduces a novel control function approach to correct position bias in implicit feedback data for learning-to-rank systems, improving ranking accuracy without requiring knowledge of click models.

Contribution

It proposes a two-stage control function method that corrects position bias without prior click model knowledge and supports nonlinear ranking models.

Findings

Outperforms existing methods in position bias correction

Does not require click or propensity model knowledge

Effectively debiases validation clicks for hyperparameter tuning

Abstract

Implicit feedback data, such as user clicks, is commonly used in learning-to-rank (LTR) systems because it is easy to collect and it often reflects user preferences. However, this data is prone to various biases, and training an LTR algorithm directly on biased data can result in suboptimal ranking performance. One of the most prominent and well-studied biases in implicit feedback data is position bias, which occurs because users are more likely to interact with higher-ranked items regardless of their true relevance. In this paper, we propose a novel control function-based method that accounts for position bias in a two-stage process. The first stage uses exogenous variation from the residuals of the ranking process to correct for position bias in the second stage click equation. Unlike previous position bias correction methods, our method does not require knowledge of the click or propensity model and allows for nonlinearity in the underlying ranking model. Moreover, our method is general and allows for debiasing any state-of-the-art ranking algorithm by plugging it into the second stage. We also introduce a new technique to debias validation clicks for hyperparameter tuning to select the optimal model in the absence of unbiased validation data. Experimental results show that our method outperforms state-of-the-art approaches in correcting for position bias.