Conflict-Aware Soft Prompting for Retrieval-Augmented Generation

TL;DR

This paper introduces CARE, a method that improves retrieval-augmented generation by detecting and mitigating conflicts between external retrieved context and the model's internal knowledge, enhancing trustworthiness.

Contribution

The paper proposes a novel conflict-aware framework with a context assessor and soft prompting to address context-memory conflicts in RAG systems.

Findings

CARE reduces conflict-related errors in QA and fact-checking tasks.

Achieves an average of 5.0% performance improvement over baseline models.

Demonstrates effectiveness across multiple benchmarks.

Abstract

Retrieval-augmented generation (RAG) enhances the capabilities of large language models (LLMs) by incorporating external knowledge into their input prompts. However, when the retrieved context contradicts the LLM's parametric knowledge, it often fails to resolve the conflict between incorrect external context and correct parametric knowledge, known as context-memory conflict. To tackle this problem, we introduce Conflict-Aware REtrieval-Augmented Generation (CARE), consisting of a context assessor and a base LLM. The context assessor encodes compact memory token embeddings from raw context tokens. Through grounded/adversarial soft prompting, the context assessor is trained to discern unreliable context and capture a guidance signal that directs reasoning toward the more reliable knowledge source. Extensive experiments show that CARE effectively mitigates context-memory conflicts, leading to an average performance gain of 5.0\% on QA and fact-checking benchmarks, establishing a promising direction for trustworthy and adaptive RAG systems.