Black-box language model explanation by context length probing

TL;DR

This paper introduces context length probing, a new model-agnostic method for explaining large language models by analyzing how predictions change with varying context lengths, providing insights into long-range dependencies.

Contribution

The paper proposes a novel explanation technique for causal language models that tracks prediction changes with context length without needing internal model access.

Findings

Effective in analyzing long-range dependencies

Model-agnostic and requires only token probabilities

Provides initial insights into model behavior

Abstract

The increasingly widespread adoption of large language models has highlighted the need for improving their explainability. We present context length probing, a novel explanation technique for causal language models, based on tracking the predictions of a model as a function of the length of available context, and allowing to assign differential importance scores to different contexts. The technique is model-agnostic and does not rely on access to model internals beyond computing token-level probabilities. We apply context length probing to large pre-trained language models and offer some initial analyses and insights, including the potential for studying long-range dependencies. The source code and an interactive demo of the method are available.