Mitigating Context-Memory Conflicts in LLMs through Dynamic Cognitive Reconciliation Decoding

TL;DR

This paper introduces DCRD, a two-stage decoding method for large language models that predicts and mitigates context-memory conflicts, improving accuracy and efficiency especially in complex, real-world scenarios.

Contribution

The paper proposes DCRD, a novel dynamic decoding approach that analyzes attention maps to predict conflicts and adapt decoding paths, enhancing conflict mitigation in LLMs.

Findings

DCRD outperforms baseline methods on six QA datasets.

Constructed ConflictKG benchmark to evaluate conflict mitigation.

DCRD maintains high accuracy and efficiency in conflict scenarios.

Abstract

Large language models accumulate extensive parametric knowledge through pre-training. However, knowledge conflicts occur when outdated or incorrect parametric knowledge conflicts with external knowledge in the context. Existing methods address knowledge conflicts through contrastive decoding, but in conflict-free scenarios, static approaches disrupt output distribution. Other dynamic decoding methods attempt to measure the degree of conflict but still struggle with complex real-world situations. In this paper, we propose a two-stage decoding method called Dynamic Cognitive Reconciliation Decoding (DCRD), to predict and mitigate context-memory conflicts. DCRD first analyzes the attention map to assess context fidelity and predict potential conflicts. Based on this prediction, the input is directed to one of two decoding paths: (1) greedy decoding, or (2) context fidelity-based dynamic decoding. This design enables DCRD to handle conflicts efficiently while maintaining high accuracy and decoding efficiency in conflict-free cases. Additionally, to simulate scenarios with frequent knowledge updates, we constructed ConflictKG, a knowledge conflict QA benchmark. Experiments on four LLMs across six QA datasets show that DCRD outperforms all baselines, achieving state-of-the-art performance.