A Survey of Model Architectures in Information Retrieval

TL;DR

This survey reviews recent advances in model architectures for information retrieval, emphasizing transformer-based models and large language models, and discusses future challenges and directions in the field.

Contribution

It provides a comprehensive overview of the evolution of IR architectures, separating structural innovations from training methods, and highlights emerging research challenges.

Findings

Transformer architectures revolutionized IR systems.

Large language models excel in zero-shot and reasoning tasks.

Future directions include efficiency, multimodal data, and autonomous search.

Abstract

The period from 2019 to the present marks one of the most significant paradigm shifts in information retrieval (IR) and natural language processing (NLP), culminating in the emergence of powerful large language models (LLMs) from 2022 onward. Methods based on pretrained encoder-only architectures (e.g., BERT) as well as decoder-only generative LLMs have outperformed many earlier approaches, demonstrating particularly strong performance in zero-shot scenarios and complex reasoning tasks. This survey examines the evolution of model architectures in IR, with a focus on two key aspects: backbone models for feature extraction and end-to-end system architectures for relevance estimation. To maintain analytical clarity, we deliberately separate architectural design from training methodologies, enabling a focused examination of structural innovations in IR systems. We trace the progression from traditional term-based retrieval models to modern neural approaches, highlighting the transformative impact of transformer-based architectures and subsequent LLM developments. The survey concludes with a forward-looking discussion of open challenges and emerging research directions, including architectural optimization for efficiency and scalability, robust handling of multimodal and multilingual data, and adaptation to novel application domains such as autonomous search agents, which may represent the next paradigm in IR.