When Abundance Conceals Weakness: Knowledge Conflict in Multilingual Models

TL;DR

This paper introduces CLEAR, a framework for systematically evaluating how multilingual LLMs resolve cross-lingual knowledge conflicts, revealing that resource abundance and linguistic affinity influence conflict resolution differently across tasks.

Contribution

The paper presents a novel evaluation framework and multilingual benchmarks to analyze conflict resolution in multilingual LLMs, highlighting task-dependent behaviors.

Findings

High-resource languages dominate reasoning tasks.

Linguistic affinity influences factual conflict resolution.

Multilingual models exhibit task-dependent conflict resolution strategies.

Abstract

Large Language Models (LLMs) encode vast world knowledge across multiple languages, yet their internal beliefs are often unevenly distributed across linguistic spaces. When external evidence contradicts these language-dependent memories, models encounter \emph{cross-lingual knowledge conflict}, a phenomenon largely unexplored beyond English-centric settings. We introduce \textbf{CLEAR}, a \textbf{C}ross-\textbf{L}ingual knowl\textbf{E}dge conflict ev\textbf{A}luation f\textbf{R}amework that systematically examines how multilingual LLMs reconcile conflicting internal beliefs and multilingual external evidence. CLEAR decomposes conflict resolution into four progressive scenarios, from multilingual parametric elicitation to competitive multi-source cross-lingual induction, and systematically evaluates model behavior across two complementary QA benchmarks with distinct task characteristics. We construct multilingual versions of ConflictQA and ConflictingQA covering 10 typologically diverse languages and evaluate six representative LLMs. Our experiments reveal a task-dependent decision dichotomy. In reasoning-intensive tasks, conflict resolution is dominated by language resource abundance, with high-resource languages exerting stronger persuasive power. In contrast, for entity-centric factual conflicts, linguistic affinity, not resource scale, becomes decisive, allowing low-resource but linguistically aligned languages to outperform distant high-resource ones.