Keyword search is all you need: Achieving RAG-Level Performance without vector databases using agentic tool use
Shreyas Subramanian, Adewale Akinfaderin, Yanyan Zhang, Ishan Singh, Mani Khanuja, Sandeep Singh, Maira Ladeira Tanke

TL;DR
This paper demonstrates that simple keyword search tools within agentic frameworks can achieve over 90% of RAG system performance, offering a cost-effective alternative without relying on vector databases.
Contribution
It provides a systematic comparison showing that keyword search can replace vector databases in RAG, simplifying implementation and reducing costs.
Findings
Keyword search achieves over 90% of RAG performance.
Agentic keyword search is simpler and more cost-effective.
Effective for scenarios with frequent knowledge base updates.
Abstract
While Retrieval-Augmented Generation (RAG) has proven effective for generating accurate, context-based responses based on existing knowledge bases, it presents several challenges including retrieval quality dependencies, integration complexity and cost. Recent advances in agentic-RAG and tool-augmented LLM architectures have introduced alternative approaches to information retrieval and processing. We question how much additional value vector databases and semantic search bring to RAG over simple, agentic keyword search in documents for question-answering. In this study, we conducted a systematic comparison between RAG-based systems and tool-augmented LLM agents, specifically evaluating their retrieval mechanisms and response quality when the agent only has access to basic keyword search tools. Our empirical analysis demonstrates that tool-based keyword search implementations within an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInformation Retrieval and Search Behavior · Topic Modeling · Biomedical Text Mining and Ontologies
