Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method

TL;DR

This paper introduces ConflictQA, a benchmark for evaluating LLM reasoning with conflicting knowledge sources, and proposes XoT, a framework to improve reasoning accuracy over heterogeneous evidence.

Contribution

It presents a new benchmark for cross-source knowledge conflicts and a novel explanation-based reasoning framework to enhance LLM faithfulness.

Findings

LLMs often fail to identify reliable evidence in conflicting knowledge scenarios.

LLMs tend to rely on either KG or textual evidence, leading to incorrect answers.

XoT improves reasoning accuracy over conflicting evidence sources.

Abstract

Large language models (LLMs) have achieved remarkable success across a wide range of applications especially when augmented by external knowledge through retrieval-augmented generation (RAG). Despite their widespread adoption, recent studies have shown that LLMs often struggle to perform faithful reasoning when conflicting knowledge is retrieved. However, existing work primarily focuses on conflicts between external knowledge and the parametric knowledge of LLMs, leaving conflicts across external knowledge largely unexplored. Meanwhile, modern RAG systems increasingly emphasize the integration of unstructured text and (semi-)structured data like knowledge graphs (KGs) to improve knowledge completeness and reasoning faithfulness. To address this gap, we introduce ConflictQA, a novel benchmark that systematically instantiates conflicts between textual evidence and KG evidence. Extensive evaluations across representative LLMs reveal that, facing such cross-source conflicts, LLMs often fail to identify reliable evidence for correct reasoning. Instead, LLMs become more sensitive to prompting choices and tend to rely exclusively on either KG or textual evidence, resulting in incorrect responses. Based on these findings, we further propose XoT, a two-stage explanation-based thinking framework tailored for reasoning over heterogeneous conflicting evidence, and verify its effectiveness with extensive experiments.