Supercharging Federated Intelligence Retrieval

TL;DR

This paper introduces a secure federated retrieval-augmented generation system that maintains confidentiality across private data silos and enhances remote LLM inference with a cascading approach involving third-party models.

Contribution

It presents a novel federated RAG framework using Flower, enabling confidential document retrieval and inference across distributed data silos with a cascading inference method.

Findings

Secure federated RAG system demonstrated

Confidential remote inference enabled in distributed environments

Cascading inference improves context incorporation

Abstract

RAG typically assumes centralized access to documents, which breaks down when knowledge is distributed across private data silos. We propose a secure Federated RAG system built using Flower that performs local silo retrieval, while server-side aggregation and text generation run inside an attested, confidential compute environment, enabling confidential remote LLM inference even in the presence of honest-but-curious or compromised servers. We also propose a cascading inference approach that incorporates a non-confidential third-party model (e.g., Amazon Nova) as auxiliary context without weakening confidentiality.