Semantic Models for the First-stage Retrieval: A Comprehensive Review

TL;DR

This paper reviews the development of semantic models for first-stage retrieval in search systems, highlighting their evolution from classical methods to neural approaches and discussing future challenges and directions.

Contribution

It provides a comprehensive survey of first-stage semantic retrieval models, unifying various approaches and analyzing their connections and differences.

Findings

Semantic models improve recall in first-stage retrieval.

Neural semantic retrieval methods are rapidly evolving.

Open challenges include efficiency and robustness of models.

Abstract

Multi-stage ranking pipelines have been a practical solution in modern search systems, where the first-stage retrieval is to return a subset of candidate documents, and latter stages attempt to re-rank those candidates. Unlike re-ranking stages going through quick technique shifts during past decades, the first-stage retrieval has long been dominated by classical term-based models. Unfortunately, these models suffer from the vocabulary mismatch problem, which may block re-ranking stages from relevant documents at the very beginning. Therefore, it has been a long-term desire to build semantic models for the first-stage retrieval that can achieve high recall efficiently. Recently, we have witnessed an explosive growth of research interests on the first-stage semantic retrieval models. We believe it is the right time to survey current status, learn from existing methods, and gain some insights for future development. In this paper, we describe the current landscape of the first-stage retrieval models under a unified framework to clarify the connection between classical term-based retrieval methods, early semantic retrieval methods and neural semantic retrieval methods. Moreover, we identify some open challenges and envision some future directions, with the hope of inspiring more researches on these important yet less investigated topics.