Zero-Shot Listwise Document Reranking with a Large Language Model

TL;DR

This paper introduces LRL, a zero-shot listwise reranker using large language models that improves document ranking without task-specific training, outperforming existing methods in web search and multilingual datasets.

Contribution

The paper presents a novel zero-shot listwise reranking approach with large language models, eliminating the need for labeled training data and enhancing ranking effectiveness.

Findings

Outperforms zero-shot pointwise rerankers on TREC datasets

Effective as a final-stage reranker for top-ranked results

Shows potential for multilingual retrieval tasks

Abstract

Supervised ranking methods based on bi-encoder or cross-encoder architectures have shown success in multi-stage text ranking tasks, but they require large amounts of relevance judgments as training data. In this work, we propose Listwise Reranker with a Large Language Model (LRL), which achieves strong reranking effectiveness without using any task-specific training data. Different from the existing pointwise ranking methods, where documents are scored independently and ranked according to the scores, LRL directly generates a reordered list of document identifiers given the candidate documents. Experiments on three TREC web search datasets demonstrate that LRL not only outperforms zero-shot pointwise methods when reranking first-stage retrieval results, but can also act as a final-stage reranker to improve the top-ranked results of a pointwise method for improved efficiency. Additionally, we apply our approach to subsets of MIRACL, a recent multilingual retrieval dataset, with results showing its potential to generalize across different languages.