Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback

TL;DR

This paper introduces ANCE-PRF, a novel query encoder that leverages pseudo relevance feedback to enhance dense retrieval effectiveness by producing more semantically rich query embeddings, significantly outperforming previous models.

Contribution

The paper presents a new PRF-based query encoder that improves dense retrieval by directly learning better query embeddings from relevance signals without changing the document index.

Findings

ANCE-PRF outperforms ANCE and other dense retrieval models on multiple datasets.

The PRF encoder effectively captures relevant information while ignoring noise.

Attention mechanisms help the model focus on useful feedback documents.

Abstract

Dense retrieval systems conduct first-stage retrieval using embedded representations and simple similarity metrics to match a query to documents. Its effectiveness depends on encoded embeddings to capture the semantics of queries and documents, a challenging task due to the shortness and ambiguity of search queries. This paper proposes ANCE-PRF, a new query encoder that uses pseudo relevance feedback (PRF) to improve query representations for dense retrieval. ANCE-PRF uses a BERT encoder that consumes the query and the top retrieved documents from a dense retrieval model, ANCE, and it learns to produce better query embeddings directly from relevance labels. It also keeps the document index unchanged to reduce overhead. ANCE-PRF significantly outperforms ANCE and other recent dense retrieval systems on several datasets. Analysis shows that the PRF encoder effectively captures the relevant and complementary information from PRF documents, while ignoring the noise with its learned attention mechanism.