SLA Management in Reconfigurable Multi-Agent RAG: A Systems Approach to Question Answering

TL;DR

This paper presents a systems approach to managing multi-agent Retrieval Augmented Generation (RAG) systems for question answering, enabling dynamic reconfiguration to meet diverse SLAs involving quality, cost, and latency.

Contribution

It introduces a method to incorporate non-functional requirements into multi-agent RAG systems, allowing for real-time reconfiguration to optimize performance under resource constraints.

Findings

Effective management of trade-offs between answer quality and cost.

Dynamic re-orchestration improves SLA compliance in QA systems.

System demonstrates adaptability to different query types and operational conditions.

Abstract

Retrieval Augmented Generation (RAG) enables Large Language Models (LLMs) to generalize to new information by decoupling reasoning capabilities from static knowledge bases. Traditional RAG enhancements have explored vertical scaling-assigning subtasks to specialized modules-and horizontal scaling-replicating tasks across multiple agents-to improve performance. However, real-world applications impose diverse Service Level Agreements (SLAs) and Quality of Service (QoS) requirements, involving trade-offs among objectives such as reducing cost, ensuring answer quality, and adhering to specific operational constraints. In this work, we present a systems-oriented approach to multi-agent RAG tailored for real-world Question Answering (QA) applications. By integrating task-specific non-functional requirements-such as answer quality, cost, and latency-into the system, we enable dynamic reconfiguration to meet diverse SLAs. Our method maps these Service Level Objectives (SLOs) to system-level parameters, allowing the generation of optimal results within specified resource constraints. We conduct a case study in the QA domain, demonstrating how dynamic re-orchestration of a multi-agent RAG system can effectively manage the trade-off between answer quality and cost. By adjusting the system based on query intent and operational conditions, we systematically balance performance and resource utilization. This approach allows the system to meet SLOs for various query types, showcasing its practicality for real-world applications.