Federated Retrieval Augmented Generation for Multi-Product Question Answering

TL;DR

This paper introduces MKP-QA, a federated retrieval-augmented generation framework for multi-product question answering, improving search accuracy and response quality across diverse enterprise domains.

Contribution

The paper proposes a novel probabilistic federated search method for multi-product RAG-QA and provides new datasets for benchmarking in this domain.

Findings

MKP-QA outperforms existing methods in retrieval accuracy.

Enhanced response quality in multi-domain question answering.

New datasets facilitate benchmarking for multi-product QA.

Abstract

Recent advancements in Large Language Models and Retrieval-Augmented Generation have boosted interest in domain-specific question-answering for enterprise products. However, AI Assistants often face challenges in multi-product QA settings, requiring accurate responses across diverse domains. Existing multi-domain RAG-QA approaches either query all domains indiscriminately, increasing computational costs and LLM hallucinations, or rely on rigid resource selection, which can limit search results. We introduce MKP-QA, a novel multi-product knowledge-augmented QA framework with probabilistic federated search across domains and relevant knowledge. This method enhances multi-domain search quality by aggregating query-domain and query-passage probabilistic relevance. To address the lack of suitable benchmarks for multi-product QAs, we also present new datasets focused on three Adobe products: Adobe Experience Platform, Target, and Customer Journey Analytics. Our experiments show that MKP-QA significantly boosts multi-product RAG-QA performance in terms of both retrieval accuracy and response quality.