GeAR: Generation Augmented Retrieval

TL;DR

GeAR is a novel retrieval method that enhances global similarity with contrastive learning and generates relevant context to capture fine-grained semantic relationships, improving interpretability and performance without extra computational cost.

Contribution

Introduces GeAR, a retrieval approach combining contrastive learning with generation modules to improve interpretability and fine-grained semantic understanding.

Findings

Achieves competitive retrieval performance across various tasks.

Provides qualitative insights into retrieval result interpretation.

Does not increase computational cost when used as a retriever.

Abstract

Document retrieval techniques are essential for developing large-scale information systems. The common approach involves using a bi-encoder to compute the semantic similarity between a query and documents. However, the scalar similarity often fail to reflect enough information, hindering the interpretation of retrieval results. In addition, this process primarily focuses on global semantics, overlooking the finer-grained semantic relationships between the query and the document's content. In this paper, we introduce a novel method, $\textbf{Ge}$neration $\textbf{A}$ugmented $\textbf{R}$etrieval ($\textbf{GeAR}$), which not only improves the global document-query similarity through contrastive learning, but also integrates well-designed fusion and decoding modules. This enables GeAR to generate relevant context within the documents based on a given query, facilitating learning to retrieve local fine-grained information. Furthermore, when used as a retriever, GeAR does not incur any additional computational cost over bi-encoders. GeAR exhibits competitive retrieval performance across diverse scenarios and tasks. Moreover, qualitative analysis and the results generated by GeAR provide novel insights into the interpretation of retrieval results. The code, data, and models will be released at \href{https://github.com/microsoft/LMOps}{https://github.com/microsoft/LMOps}.