Resolving Knowledge Conflicts in Large Language Models

TL;DR

This paper evaluates how well large language models handle knowledge conflicts, introducing a framework to assess their abilities to identify, locate, and respond to conflicting information, and proposes instruction-based improvements.

Contribution

It provides a comprehensive evaluation framework for knowledge conflicts in LLMs and introduces new instruction-based methods to improve their conflict resolution capabilities.

Findings

LLMs excel at detecting conflicts but struggle to pinpoint specific conflicting information.

LLMs often produce inconsistent responses in conflicting scenarios.

Knowledge domain significantly impacts LLMs' ability to resolve conflicts.

Abstract

Large language models (LLMs) often encounter knowledge conflicts, scenarios where discrepancy arises between the internal parametric knowledge of LLMs and non-parametric information provided in the prompt context. In this work we ask what are the desiderata for LLMs when a knowledge conflict arises and whether existing LLMs fulfill them. We posit that LLMs should 1) identify knowledge conflicts, 2) pinpoint conflicting information segments, and 3) provide distinct answers or viewpoints in conflicting scenarios. To this end, we introduce an evaluation framework for simulating contextual knowledge conflicts and quantitatively evaluating to what extent LLMs achieve these goals. It includes diverse and complex situations of knowledge conflict, knowledge from diverse entities and domains, two synthetic conflict creation methods, and settings with progressively increasing difficulty to reflect realistic knowledge conflicts. Extensive experiments with the framework reveal that while LLMs perform well in identifying the existence of knowledge conflicts, they struggle to determine the specific conflicting knowledge and produce a response with distinct answers amidst conflicting information. To address these challenges, we propose new instruction-based approaches that augment LLMs to better achieve the three goals. Further analysis shows that abilities to tackle knowledge conflicts are greatly impacted by factors such as knowledge domain, while generating robust responses to knowledge conflict scenarios remains an open research question.