HASH-RAG: Bridging Deep Hashing with Retriever for Efficient, Fine Retrieval and Augmented Generation

TL;DR

Hash-RAG introduces a deep hashing-based retrieval framework that significantly reduces retrieval time and computational costs in retrieval-augmented generation, while maintaining high accuracy and contextual relevance.

Contribution

The paper presents a novel hash-based retrieval method integrated with prompt-guided chunking to improve efficiency and effectiveness in retrieval-augmented generation tasks.

Findings

Achieves 90% reduction in retrieval time compared to traditional methods.

Maintains high recall performance in large knowledge bases.

Outperforms baseline methods by 1.4-4.3% in EM scores.

Abstract

Retrieval-Augmented Generation (RAG) encounters efficiency challenges when scaling to massive knowledge bases while preserving contextual relevance. We propose Hash-RAG, a framework that integrates deep hashing techniques with systematic optimizations to address these limitations. Our queries directly learn binary hash codes from knowledgebase code, eliminating intermediate feature extraction steps, and significantly reducing storage and computational overhead. Building upon this hash-based efficient retrieval framework, we establish the foundation for fine-grained chunking. Consequently, we design a Prompt-Guided Chunk-to-Context (PGCC) module that leverages retrieved hash-indexed propositions and their original document segments through prompt engineering to enhance the LLM's contextual awareness. Experimental evaluations on NQ, TriviaQA, and HotpotQA datasets demonstrate that our approach achieves a 90% reduction in retrieval time compared to conventional methods while maintaining considerate recall performance. Additionally, The proposed system outperforms retrieval/non-retrieval baselines by 1.4-4.3% in EM scores.