Revisiting Bi-Encoder Neural Search: An Encoding--Searching Separation Perspective

TL;DR

This paper critically examines the bi-encoder neural search architecture, identifies its limitations, and introduces an encoding--searching separation perspective to improve understanding and performance.

Contribution

It proposes a new conceptual framework that separates encoding and searching, addressing existing issues and guiding future research in neural search architectures.

Findings

Identifies encoding bottleneck and embedding search limitations.

Proposes the encoding--searching separation perspective.

Suggests strategies to mitigate issues and improve retrieval performance.

Abstract

This paper reviews, analyzes, and proposes a new perspective on the bi-encoder architecture for neural search. While the bi-encoder architecture is widely used due to its simplicity and scalability at test time, it has some notable issues such as low performance on seen datasets and weak zero-shot performance on new datasets. In this paper, we analyze these issues and summarize two main critiques: the encoding information bottleneck problem and limitations of the basic assumption of embedding search. We then construct a thought experiment to logically analyze the encoding and searching operations and challenge the basic assumptions of embedding search. Building on these observations, we propose a new perspective on the bi-encoder architecture called the \textit{encoding--searching separation} perspective, which conceptually and practically separates the encoding and searching operations. This framework is applied to explain the root cause of existing issues and suggest mitigation strategies, potentially lowering training costs and improving retrieval performance. Finally, we discuss the broader implications of the ideas underlying this perspective, the new design surface it exposes, and potential research directions arising from it.