ConflictRAG: Detecting and Resolving Knowledge Conflicts in Retrieval Augmented Generation

TL;DR

ConflictRAG introduces a framework for detecting, classifying, and resolving knowledge conflicts in retrieval-augmented generation systems, improving accuracy and reducing costs.

Contribution

It presents a novel two-stage conflict detection method, a data-driven source credibility assessment, and a diagnostic score for conflict handling in RAG.

Findings

Achieved 90.8% conflict detection accuracy.

Reduced API costs by 62%.

Improved correctness by 5.3-6.1% over baselines.

Abstract

Retrieval-Augmented Generation (RAG) systems implicitly assume mutual consistency among retrieved documents -- an assumption that frequently fails in practice. We present ConflictRAG, a conflict-aware RAG framework that detects, classifies, and resolves knowledge conflicts prior to answer generation. The framework introduces three contributions: (1) a two-stage conflict detection module combining a lightweight embedding-based MLP classifier with selective LLM refinement, reducing API costs by 62% while maintaining 90.8% detection accuracy; (2) an Entropy-TOPSIS framework for data-driven source credibility assessment, improving selection accuracy by 7.1% over manual heuristics; and (3) a Conflict-Aware RAG Score (CARS) for diagnostic evaluation of conflict-handling capabilities. Experiments on three benchmarks against six baselines demonstrate 88.7% conflict-detection F1 and consistent 5.3--6.1% correctness gains over the strongest conflict-aware baseline, with the pipeline transferring effectively across backbone LLMs.