Towards Two-Stage Counterfactual Learning to Rank

TL;DR

This paper introduces a novel two-stage counterfactual learning to rank method that jointly trains candidate generators and rankers, addressing scalability issues in large document sets and improving ranking performance.

Contribution

It proposes the first joint CLTR estimator and learning method for two-stage ranking systems, considering interactions between candidate generation and ranking.

Findings

Joint CLTR method outperforms baselines in semi-synthetic benchmarks.

Effective for large-scale document ranking systems.

Addresses bias correction in two-stage ranking pipelines.

Abstract

Counterfactual learning to rank (CLTR) aims to learn a ranking policy from user interactions while correcting for the inherent biases in interaction data, such as position bias. Existing CLTR methods assume a single ranking policy that selects top-K ranking from the entire document candidate set. In real-world applications, the candidate document set is on the order of millions, making a single-stage ranking policy impractical. In order to scale to millions of documents, real-world ranking systems are designed in a two-stage fashion, with a candidate generator followed by a ranker. The existing CLTR method for a two-stage offline ranking system only considers the top-1 ranking set-up and only focuses on training the candidate generator, with the ranker fixed. A CLTR method for training both the ranker and candidate generator jointly is missing from the existing literature. In this paper, we propose a two-stage CLTR estimator that considers the interaction between the two stages and estimates the joint value of the two policies offline. In addition, we propose a novel joint optimization method to train the candidate and ranker policies, respectively. To the best of our knowledge, we are the first to propose a CLTR estimator and learning method for two-stage ranking. Experimental results on a semi-synthetic benchmark demonstrate the effectiveness of the proposed joint CLTR method over baselines.