AgenticRAG: Agentic Retrieval for Enterprise Knowledge Bases

TL;DR

AgenticRAG enhances enterprise knowledge base retrieval by enabling language models to autonomously search, analyze, and navigate documents, significantly improving accuracy and efficiency over traditional methods.

Contribution

Introduces a lightweight agentic harness that layers on top of existing search infrastructure, allowing iterative retrieval and analysis with substantial performance gains.

Findings

49.6% recall@1 on BRIGHT benchmark (+21.8 pp over baseline)

0.96 factuality score on WixQA (+13% relative improvement)

92% answer correctness on FinanceBench, close to oracle evidence

Abstract

We present AgenticRAG, a practical agentic harness for retrieval and analysis over enterprise knowledge bases. Standard RAG pipelines place significant burden of grounding on the search stack, constraining the language model to a fixed candidate set chosen deep in the retrieval process. Our approach reduces this overdependence by layering a lightweight harness on top of existing enterprise search infrastructure, equipping a reasoning LLM with search, find, open, and summarize tools enabling the model to iteratively retrieve information, navigate within documents, and analyze evidence autonomously. On three open benchmarks we observe substantial gains: $49.6\%$ recall@1 on BRIGHT (+21.8 pp over the best embedding baseline), 0.96 factuality on WixQA ($+13\%$ relative improvement), and $92\%$ answer correctness on FinanceBench--within 2 pp of oracle access to true evidence. Ablation studies show that the most significant factor is the shift from single-shot retrieval to agentic tool use ($5.9\times$ improvement), while multi-query search and in-document navigation contribute to both quality and efficiency. We present various design choices in our agentic harness that were informed by pre-production deployments. Our results demonstrate its suitability for real-world enterprise production environments.