Generative Relevance Feedback and Convergence of Adaptive Re-Ranking: University of Glasgow Terrier Team at TREC DL 2023

TL;DR

This paper explores the use of generative relevance feedback from large language models combined with adaptive re-ranking techniques to improve retrieval performance in the TREC Deep Learning Track, demonstrating some performance gains.

Contribution

It introduces a novel combination of generative relevance feedback with adaptive re-ranking over a corpus graph, analyzing their joint effect on retrieval effectiveness.

Findings

Generative relevance feedback yields performance improvements.

Adaptive re-ranking's effectiveness depends on first-stage retrieval quality.

Combining both methods achieves the best retrieval results.

Abstract

This paper describes our participation in the TREC 2023 Deep Learning Track. We submitted runs that apply generative relevance feedback from a large language model in both a zero-shot and pseudo-relevance feedback setting over two sparse retrieval approaches, namely BM25 and SPLADE. We couple this first stage with adaptive re-ranking over a BM25 corpus graph scored using a monoELECTRA cross-encoder. We investigate the efficacy of these generative approaches for different query types in first-stage retrieval. In re-ranking, we investigate operating points of adaptive re-ranking with different first stages to find the point in graph traversal where the first stage no longer has an effect on the performance of the overall retrieval pipeline. We find some performance gains from the application of generative query reformulation. However, our strongest run in terms of P@10 and nDCG@10 applied both adaptive re-ranking and generative pseudo-relevance feedback, namely uogtr_b_grf_e_gb.