IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neurons

TL;DR

This paper introduces IRCAN, a framework that identifies and reweights context-aware neurons in LLMs to resolve knowledge conflicts, improving context-sensitive generation without retraining the models.

Contribution

IRCAN is a novel method that enhances LLMs' ability to handle conflicting knowledge by focusing on neurons critical for context processing, a scalable and plug-and-play solution.

Findings

IRCAN significantly reduces knowledge conflicts in LLM outputs.

The method improves context sensitivity across various models and tasks.

IRCAN is easy to integrate with existing LLM architectures.

Abstract

It is widely acknowledged that large language models (LLMs) encode a vast reservoir of knowledge after being trained on mass data. Recent studies disclose knowledge conflicts in LLM generation, wherein outdated or incorrect parametric knowledge (i.e., encoded knowledge) contradicts new knowledge provided in the context. To mitigate such knowledge conflicts, we propose a novel framework, IRCAN (Identifying and Reweighting Context-Aware Neurons) to capitalize on neurons that are crucial in processing contextual cues. Specifically, IRCAN first identifies neurons that significantly contribute to context processing, utilizing a context-aware attribution score derived from integrated gradients. Subsequently, the identified context-aware neurons are strengthened via reweighting. In doing so, we steer LLMs to generate context-sensitive outputs with respect to the new knowledge provided in the context. Extensive experiments conducted across a variety of models and tasks demonstrate that IRCAN not only achieves remarkable improvements in handling knowledge conflicts but also offers a scalable, plug-and-play solution that can be integrated seamlessly with existing models. Our codes are released at https://github.com/danshi777/IRCAN.