Beyond Precision: A Study on Recall of Initial Retrieval with Neural Representations

TL;DR

This paper investigates how neural representations can enhance the recall of initial document retrieval in information retrieval systems, addressing vocabulary mismatch issues and proposing hybrid search schemes to improve initial recall with minimal overhead.

Contribution

It introduces a neural index and two hybrid search schemes that combine neural and symbolic indices to improve initial retrieval recall in IR systems.

Findings

Hybrid schemes improve recall with small overhead

Neural index enhances initial retrieval effectiveness

Parallel and sequential search schemes outperform traditional methods

Abstract

Vocabulary mismatch is a central problem in information retrieval (IR), i.e., the relevant documents may not contain the same (symbolic) terms of the query. Recently, neural representations have shown great success in capturing semantic relatedness, leading to new possibilities to alleviate the vocabulary mismatch problem in IR. However, most existing efforts in this direction have been devoted to the re-ranking stage. That is to leverage neural representations to help re-rank a set of candidate documents, which are typically obtained from an initial retrieval stage based on some symbolic index and search scheme (e.g., BM25 over the inverted index). This naturally raises a question: if the relevant documents have not been found in the initial retrieval stage due to vocabulary mismatch, there would be no chance to re-rank them to the top positions later. Therefore, in this paper, we study the problem how to employ neural representations to improve the recall of relevant documents in the initial retrieval stage. Specifically, to meet the efficiency requirement of the initial stage, we introduce a neural index for the neural representations of documents, and propose two hybrid search schemes based on both neural and symbolic indices, namely the parallel search scheme and the sequential search scheme. Our experiments show that both hybrid index and search schemes can improve the recall of the initial retrieval stage with small overhead.