Counterfactual Query Rewriting to Use Historical Relevance Feedback

TL;DR

This paper introduces methods to rewrite user queries using historical relevance feedback, improving retrieval effectiveness even when documents have changed or are unavailable, outperforming some transformer-based approaches.

Contribution

It proposes query rewriting techniques that leverage past relevance feedback to enhance retrieval, addressing challenges of document changes and unavailability.

Findings

Query rewriting with historical feedback improves retrieval results.

Rewritten queries outperform some transformer-based methods.

Approaches are validated in the CLEF LongEval scenario.

Abstract

When a retrieval system receives a query it has encountered before, previous relevance feedback, such as clicks or explicit judgments can help to improve retrieval results. However, the content of a previously relevant document may have changed, or the document might not be available anymore. Despite this evolved corpus, we counterfactually use these previously relevant documents as relevance signals. In this paper we proposed approaches to rewrite user queries and compare them against a system that directly uses the previous qrels for the ranking. We expand queries with terms extracted from the previously relevant documents or derive so-called keyqueries that rank the previously relevant documents to the top of the current corpus. Our evaluation in the CLEF LongEval scenario shows that rewriting queries with historical relevance feedback improves the retrieval effectiveness and even outperforms computationally expensive transformer-based approaches.