SPD-RAG: Sub-Agent Per Document Retrieval-Augmented Generation

TL;DR

SPD-RAG introduces a hierarchical multi-agent framework for multi-document question answering, decomposing the task across documents to improve answer quality and scalability while reducing costs.

Contribution

It presents a novel hierarchical multi-agent approach with document-level specialization and centralized fusion for efficient, high-quality multi-document QA.

Findings

Outperforms baseline models on LOONG benchmark with higher scores.

Achieves 58.1 average score, significantly better than RAG and Agentic RAG.

Reduces API cost to 38% of full-context baseline.

Abstract

Answering complex, real-world queries often requires synthesizing facts scattered across vast document corpora. In these settings, standard retrieval-augmented generation (RAG) pipelines suffer from incomplete evidence coverage, while long-context large language models (LLMs) struggle to reason reliably over massive inputs. We introduce SPD-RAG, a hierarchical multi-agent framework for exhaustive cross-document question answering that decomposes the problem along the document axis. Each document is processed by a dedicated document-level agent operating only on its own content, enabling focused retrieval, while a coordinator dispatches tasks to relevant agents and aggregates their partial answers. Agent outputs are synthesized by merging partial answers through a token-bounded synthesis layer (which supports recursive map-reduce for massive corpora). This document-level specialization with centralized fusion improves scalability and answer quality in heterogeneous multidocument settings while yielding a modular, extensible retrieval pipeline. On the LOONG benchmark (EMNLP 2024) for long-context multi-document QA, SPD-RAG achieves an Avg Score of 58.1 (GPT-5 evaluation), outperforming Normal RAG (33.0) and Agentic RAG (32.8) while using only 38% of the API cost of a full-context baseline (68.0).