Feature-Enhanced Network with Hybrid Debiasing Strategies for Unbiased Learning to Rank

TL;DR

This paper presents a feature-enhanced neural approach with hybrid debiasing strategies for unbiased learning to rank, effectively addressing click bias issues and achieving top performance in WSDM Cup 2023.

Contribution

It introduces a novel combination of heuristic feature extraction, label adjustment, propensity calibration, and model ensembling for improved unbiased ranking.

Findings

Achieved a DCG@10 score of 9.80, ranking third in WSDM Cup 2023.

Effectively mitigated position and trust biases in click data.

Demonstrated significant performance improvement over baseline methods.

Abstract

Unbiased learning to rank (ULTR) aims to mitigate various biases existing in user clicks, such as position bias, trust bias, presentation bias, and learn an effective ranker. In this paper, we introduce our winning approach for the "Unbiased Learning to Rank" task in WSDM Cup 2023. We find that the provided data is severely biased so neural models trained directly with the top 10 results with click information are unsatisfactory. So we extract multiple heuristic-based features for multi-fields of the results, adjust the click labels, add true negatives, and re-weight the samples during model training. Since the propensities learned by existing ULTR methods are not decreasing w.r.t. positions, we also calibrate the propensities according to the click ratios and ensemble the models trained in two different ways. Our method won the 3rd prize with a DCG@10 score of 9.80, which is 1.1% worse than the 2nd and 25.3% higher than the 4th.