Context Dependence and Reliability in Autoregressive Language Models

TL;DR

This paper introduces RISE, a method that improves the interpretability of large language models by accurately identifying essential context elements, reducing redundancy effects, and providing more reliable explanations for model outputs.

Contribution

The paper presents RISE, a novel approach that quantifies the unique influence of context elements in LLMs, enhancing explanation stability and interpretability.

Findings

RISE outperforms traditional explanation methods in robustness.

It effectively distinguishes essential from redundant context.

Provides more trustworthy explanations for LLM outputs.

Abstract

Large language models (LLMs) generate outputs by utilizing extensive context, which often includes redundant information from prompts, retrieved passages, and interaction history. In critical applications, it is vital to identify which context elements actually influence the output, as standard explanation methods struggle with redundancy and overlapping context. Minor changes in input can lead to unpredictable shifts in attribution scores, undermining interpretability and raising concerns about risks like prompt injection. This work addresses the challenge of distinguishing essential context elements from correlated ones. We introduce RISE (Redundancy-Insensitive Scoring of Explanation), a method that quantifies the unique influence of each input relative to others, minimizing the impact of redundancies and providing clearer, stable attributions. Experiments demonstrate that RISE offers more robust explanations than traditional methods, emphasizing the importance of conditional information for trustworthy LLM explanations and monitoring.