Distributed Retrieval-Augmented Generation

TL;DR

This paper introduces Distributed Retrieval-Augmented Generation (DRAG), a decentralized framework that enhances data privacy and scalability for knowledge retrieval in large language models, using peer-to-peer networks and a topic-aware random walk algorithm.

Contribution

The paper proposes DRAG, a novel decentralized RAG framework that eliminates the need for a centralized knowledge base and introduces TARW for efficient peer discovery.

Findings

DRAG achieves near-centralized RAG performance.

Uses half as many messages as flooding for knowledge retrieval.

Effective across diverse datasets and LLMs.

Abstract

As large language models (LLMs) become increasingly adopted on edge devices, Retrieval-Augmented Generation (RAG) is gaining prominence as a solution to address factual deficiencies and hallucinations by integrating external knowledge. However, centralized RAG architectures face significant challenges in data privacy and scalability. For instance, smart healthcare services often rely on collecting sensitive patient data and building a centralized knowledge base to provide better diagnosis and treatment advice, while privacy concerns significantly impede this process. Besides, maintaining a comprehensive and continuously updated knowledge base is costly, particularly in response to regional epidemics and rapidly mutating viruses. To address these challenges, this paper introduces Distributed Retrieval-Augmented Generation (DRAG), a novel framework that improves data privacy by eliminating the need for a centralized knowledge base and restoring data control to owners. DRAG incorporates a Topic-Aware Random Walk (TARW) algorithm that leverages LLMs to extract query topics and facilitate targeted peer discovery within a peer-to-peer network, enabling efficient knowledge retrieval in decentralized environments. Extensive experiments across three diverse datasets and LLMs demonstrate that DRAG with TARW achieves near-centralized RAG performance by using half as many messages as flooding. The code is available at https://github.com/xuchenhao001/DRAG.