Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding

TL;DR

This paper introduces Context-Fidelity Boosting (CFB), a decoding-time method inspired by watermarking that enhances faithfulness in LLM outputs by increasing source-supported token probabilities.

Contribution

It presents a novel, lightweight decoding framework with three boosting strategies that significantly improve faithfulness without retraining or architectural changes.

Findings

CFB consistently improves faithfulness metrics across tasks.

The method requires minimal additional computation during decoding.

Experiments show effectiveness across multiple open-source LLMs.

Abstract

Large language models (LLMs) often produce content that contradicts or overlooks information provided in the input context, a phenomenon known as faithfulness hallucination. In this paper, we propose Context-Fidelity Boosting (CFB), a lightweight and general decoding-time framework that reduces such hallucinations by increasing the generation probability of source-supported tokens. Motivated by logit-shaping principles from watermarking techniques, CFB applies additive token-level logit adjustments based on a token's degree of support from the input context. Specifically, we develop three boosting strategies: static boosting, which applies a fixed bias to source-supported tokens; context-aware boosting, which scales this bias using the divergence between next-token distributions with and without context; and token-aware boosting, which further redistributes the adaptive bias according to local relevance estimated from source-position attention and source-scoped semantic similarity. CFB requires no retraining or architectural changes, making it compatible with a wide range of LLMs. Experiments on summarization and question answering tasks across multiple open-source LLMs show that CFB consistently improves faithfulness metrics with minimal generation overhead. Our implementation is fully open-sourced.