Taxonomy of the Retrieval System Framework: Pitfalls and Paradigms

TL;DR

This paper presents a comprehensive taxonomy of retrieval system design, analyzing key layers and trade-offs to guide practitioners in optimizing efficiency and effectiveness in neural search systems.

Contribution

It introduces a structured framework categorizing design decisions across multiple layers, highlighting pitfalls and paradigms for improving retrieval system performance.

Findings

Analyzes how loss functions and architectures influence relevance.

Evaluates segmentation strategies for long documents.

Discusses methods to enhance robustness and domain generalization.

Abstract

Designing an embedding retrieval system requires navigating a complex design space of conflicting trade-offs between efficiency and effectiveness. This work structures these decisions as a vertical traversal of the system design stack. We begin with the Representation Layer by examining how loss functions and architectures, specifically Bi-encoders and Cross-encoders, define semantic relevance and geometric projection. Next, we analyze the Granularity Layer and evaluate how segmentation strategies like Atomic and Hierarchical chunking mitigate information bottlenecks in long-context documents. Moving to the Orchestration Layer, we discuss methods that transcend the single-vector paradigm, including hierarchical retrieval, agentic decomposition, and multi-stage reranking pipelines to resolve capacity limitations. Finally, we address the Robustness Layer by identifying architectural mitigations for domain generalization failures, lexical blind spots, and the silent degradation of retrieval quality due to temporal drift. By categorizing these limitations and design choices, we provide a comprehensive framework for practitioners to optimize the efficiency-effectiveness frontier in modern neural search systems.