GRM: Generative Relevance Modeling Using Relevance-Aware Sample Estimation for Document Retrieval

TL;DR

This paper introduces GRM, a new method that improves document retrieval by accurately weighting generated query expansion terms using relevance-aware sampling and neural re-ranking, outperforming previous approaches.

Contribution

The paper proposes Generative Relevance Modeling with Relevance-Aware Sample Estimation to enhance query expansion effectiveness in document retrieval.

Findings

GRM improves MAP by 6-9% on benchmarks.

GRM increases R@1k by 2-4%.

Outperforms previous relevance feedback methods.

Abstract

Recent studies show that Generative Relevance Feedback (GRF), using text generated by Large Language Models (LLMs), can enhance the effectiveness of query expansion. However, LLMs can generate irrelevant information that harms retrieval effectiveness. To address this, we propose Generative Relevance Modeling (GRM) that uses Relevance-Aware Sample Estimation (RASE) for more accurate weighting of expansion terms. Specifically, we identify similar real documents for each generated document and use a neural re-ranker to estimate their relevance. Experiments on three standard document ranking benchmarks show that GRM improves MAP by 6-9% and R@1k by 2-4%, surpassing previous methods.