PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoning

TL;DR

PrismRAG is a fine-tuning framework that enhances retrieval-augmented generation models by making them more resilient to distractors and better at reasoning, leading to improved factual accuracy across multiple benchmarks.

Contribution

It introduces a distractor-aware training method and reasoning-centric habits to improve RAG models without extensive human instructions.

Findings

Increases factuality by 5.4% on average across benchmarks.

Outperforms state-of-the-art RAG solutions.

Effective in diverse application scenarios.

Abstract

Retrieval-augmented generation (RAG) often falls short when retrieved context includes confusing semi-relevant passages, or when answering questions require deep contextual understanding and reasoning. We propose an efficient fine-tuning framework, called PrismRAG, that (i) trains the model with distractor-aware QA pairs mixing gold evidence with subtle distractor passages, and (ii) instills reasoning-centric habits that make the LLM plan, rationalize, and synthesize without relying on extensive human engineered instructions. Evaluated across 12 open-book RAG QA benchmarks spanning diverse application domains and scenarios, PrismRAG improves average factuality by 5.4%, outperforming state-of-the-art solutions.