No-Worse Context-Aware Decoding: Preventing Neutral Regression in Context-Conditioned Generation

TL;DR

This paper introduces NWCAD, a decoding method that prevents language models from overwriting correct answers when external context is unhelpful, ensuring reliability without sacrificing context use.

Contribution

The paper presents NWCAD, a novel decode-time adapter that mitigates neutral regression in context-conditioned generation, improving reliability while maintaining context utilization.

Findings

NWCAD prevents neutral regression on baseline-correct items.

NWCAD maintains strong accuracy on helpful contexts.

Evaluation shows improved reliability without loss of context benefits.

Abstract

Large language models (LLMs) can answer questions and summarize documents when conditioned on external contexts (e.g., retrieved evidence), yet context use remains unreliable: models may overwrite an already-correct output (neutral regression) even when the context is non-informative. We formalize neutral regression as a do-no-harm requirement and quantify it by measuring accuracy drops on baseline-correct items under answer-consistent contexts. We propose No-Worse Context-Aware Decoding (NWCAD), a decode-time adapter built on a two-stream setup with a two-stage gate: it backs off to no-context decoding when the context is non-informative, and otherwise uses context-conditioned decoding with a CAD-style fallback under uncertainty. We evaluate NWCAD on benchmarks that separate do-no-harm reliability from context utilization (accuracy gains on genuinely helpful contexts). NWCAD prevents neutral regression on baseline-correct items while preserving strong context-driven accuracy on helpful contexts.