RAQG-QPP: Query Performance Prediction with Retrieved Query Variants and Retrieval Augmented Query Generation

TL;DR

This paper introduces RAQG-QPP, a novel method that uses retrieved query variants and LLM-generated variants to improve query performance prediction, especially for neural rankers.

Contribution

It proposes leveraging retrieved queries and LLMs to generate coherent query variants, enhancing unsupervised QPP accuracy for neural retrieval models.

Findings

RAQG-QPP outperforms existing QV-based methods by up to 30% on neural models.

Using retrieved queries and LLM-generated variants improves QPP accuracy.

Experiments on TREC DL'19 and DL'20 validate the effectiveness of the approach.

Abstract

Query Performance Prediction (QPP) estimates the retrieval quality of ranking models without the use of any human-assessed relevance judgements, and finds applications in query-specific selective decision making to improve overall retrieval effectiveness. Although unsupervised QPP approaches are effective for lexical retrieval models, they usually perform weaker for neural rankers. Recent work shows that leveraging query variants (QVs), i.e., queries with potentially similar information needs to a given query, can enhance unsupervised QPP accuracy. However, existing QV-based prediction methods rely on query variants generated by term expansion of the input query, which is likely to yield incoherent, hallucinatory and off-topic QVs. In this paper, we propose to make use of queries retrieved from a log of past queries as QVs to be subsequently used for QPP. In addition to directly applying retrieved QVs in QPP, we further propose to leverage large language models (LLMs) to generate QVs conditioned on the retrieved QVs, thus mitigating the limitation of relying only on existing queries in a log. Experiments on TREC DL'19 and DL'20 show that QPP enhanced with RAQG outperform the best-performing existing QV-based prediction approach by as much as 30% on neural ranking models such as MonoT5.