Top-Down Partitioning for Efficient List-Wise Ranking
Andrew Parry, Sean MacAvaney, Debasis Ganguly

TL;DR
This paper introduces a top-down partitioning algorithm for list-wise ranking with large language models, reducing inference calls by 33% and improving parallelization over traditional sliding window methods.
Contribution
The paper proposes a novel top-down partitioning algorithm for list-wise ranking that is parallelizable and more efficient than sliding window approaches.
Findings
Reduces inference calls by approximately 33% at depth 100.
Matches the performance of prior approaches across multiple re-rankers.
Addresses limitations of sliding window methods in list-wise ranking.
Abstract
Large Language Models (LLMs) have significantly impacted many facets of natural language processing and information retrieval. Unlike previous encoder-based approaches, the enlarged context window of these generative models allows for ranking multiple documents at once, commonly called list-wise ranking. However, there are still limits to the number of documents that can be ranked in a single inference of the model, leading to the broad adoption of a sliding window approach to identify the k most relevant items in a ranked list. We argue that the sliding window approach is not well-suited for list-wise re-ranking because it (1) cannot be parallelized in its current form, (2) leads to redundant computational steps repeatedly re-scoring the best set of documents as it works its way up the initial ranking, and (3) prioritizes the lowest-ranked documents for scoring rather than the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsOptimization and Search Problems · Data Management and Algorithms · Machine Learning and Algorithms
MethodsSparse Evolutionary Training
