HaS: Accelerating RAG through Homology-Aware Speculative Retrieval

TL;DR

HaS is a homology-aware speculative retrieval framework that accelerates RAG by reducing latency with minimal accuracy loss, leveraging query homology for efficient document retrieval.

Contribution

It introduces a novel homology-aware speculative retrieval method that significantly speeds up RAG without compromising much accuracy.

Findings

Reduces retrieval latency by up to 37%.

Maintains 98-99% of original accuracy.

Effective in accelerating multi-hop queries.

Abstract

Retrieval-Augmented Generation (RAG) expands the knowledge boundary of large language models (LLMs) at inference by retrieving external documents as context. However, retrieval becomes increasingly time-consuming as the knowledge databases grow in size. Existing acceleration strategies either compromise accuracy through approximate retrieval, or achieve marginal gains by reusing results of strictly identical queries. We propose HaS, a homology-aware speculative retrieval framework that performs low-latency speculative retrieval over restricted scopes to obtain candidate documents, followed by validating whether they contain the required knowledge. The validation, grounded in the homology relation between queries, is formulated as a homologous query re-identification task: once a previously observed query is identified as a homologous re-encounter of the incoming query, the draft is deemed acceptable, allowing the system to bypass slow full-database retrieval. Benefiting from the prevalence of homologous queries under real-world popularity patterns, HaS achieves substantial efficiency gains. Extensive experiments demonstrate that HaS reduces retrieval latency by 23.74% and 36.99% across datasets with only a 1-2% marginal accuracy drop. As a plug-and-play solution, HaS also significantly accelerates complex multi-hop queries in modern agentic RAG pipelines. Source code is available at: https://github.com/ErrEqualsNil/HaS.