End-to-End Retrieval in Continuous Space

TL;DR

This paper explores end-to-end continuous space retrieval using learned embeddings and approximate nearest neighbor search, achieving significant improvements over discrete methods in question retrieval tasks.

Contribution

It introduces a method for end-to-end continuous retrieval with learned models and demonstrates improved performance over traditional discrete indexing methods.

Findings

8% MAP improvement on one task

26% MAP improvement on another

Modified datasets for better evaluation

Abstract

Most text-based information retrieval (IR) systems index objects by words or phrases. These discrete systems have been augmented by models that use embeddings to measure similarity in continuous space. But continuous-space models are typically used just to re-rank the top candidates. We consider the problem of end-to-end continuous retrieval, where standard approximate nearest neighbor (ANN) search replaces the usual discrete inverted index, and rely entirely on distances between learned embeddings. By training simple models specifically for retrieval, with an appropriate model architecture, we improve on a discrete baseline by 8% and 26% (MAP) on two similar-question retrieval tasks. We also discuss the problem of evaluation for retrieval systems, and show how to modify existing pairwise similarity datasets for this purpose.