Is Agentic RAG worth it? An experimental comparison of RAG approaches

TL;DR

This paper empirically compares Enhanced and Agentic Retrieval-Augmented Generation (RAG) approaches, analyzing their performance and costs to guide practical application choices.

Contribution

It provides a comprehensive evaluation of both RAG paradigms, highlighting their strengths, weaknesses, and suitable use cases based on empirical results.

Findings

Agentic RAG often achieves better performance in complex tasks.

Enhanced RAG can be more cost-effective for simpler queries.

Trade-offs exist between accuracy and computational cost in both approaches.

Abstract

Retrieval-Augmented Generation (RAG) systems are usually defined by the combination of a generator and a retrieval component that extracts textual context from a knowledge base to answer user queries. However, such basic implementations exhibit several limitations, including noisy or suboptimal retrieval, misuse of retrieval for out-of-scope queries, weak query-document matching, and variability or cost associated with the generator. These shortcomings have motivated the development of "Enhanced" RAG, where dedicated modules are introduced to address specific weaknesses in the workflow. More recently, the growing self-reflective capabilities of Large Language Models (LLMs) have enabled a new paradigm, often referred to as "Agentic" RAG. In this approach, an LLM orchestrates the entire process, deciding which actions to perform, when to perform them, and whether to iterate. Despite the rapid adoption of both paradigms, it remains unclear which approach is preferable under which conditions. In this work, we conduct an empirically driven evaluation of "Enhanced" and "Agentic" RAG across multiple scenarios and dimensions. Our results provide practical insights into the trade-offs between the two paradigms, offering guidance on selecting the most effective RAG design for real-world applications, considering both performance and costs.