How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs

TL;DR

This paper investigates how context influences the geometric representation of statement truth in LLMs, revealing layer-wise evolution, magnitude changes, and differences based on model size and context relevance.

Contribution

It provides the first geometric analysis of how context alters truth vectors in LLMs across layers, models, and datasets.

Findings

Truth vectors are orthogonal in early layers and converge in middle layers.

Adding context generally increases the magnitude of truth vectors.

Larger models use directional change to distinguish relevant from irrelevant context.

Abstract

Large Language Models (LLMs) often encode whether a statement is true as a vector in their residual stream activations. These vectors, also known as truth vectors, have been studied in prior work, however how they change when context is introduced remains unexplored. We study this question by measuring (1) the directional change ($\theta$) between the truth vectors with and without context and (2) the relative magnitude of the truth vectors upon adding context. Across four LLMs and four datasets, we find that (1) truth vectors are roughly orthogonal in early layers, converge in middle layers, and may stabilize or continue increasing in later layers; (2) adding context generally increases the truth vector magnitude, i.e., the separation between true and false representations in the activation space is amplified; (3) larger models distinguish relevant from irrelevant context mainly through directional change ($\theta$), while smaller models show this distinction through magnitude differences. We also find that context conflicting with parametric knowledge produces larger geometric changes than parametrically aligned context. To the best of our knowledge, this is the first work that provides a geometric characterization of how context transforms the truth vector in the activation space of LLMs.