The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors

TL;DR

This paper reveals that beliefs in large language models are geometrically encoded as curved manifolds in representation space, and introduces geometry-aware methods for more effective interventions.

Contribution

It demonstrates that LLM posteriors form curved manifolds and proposes geometry-aware interventions to better manipulate these beliefs.

Findings

Parameter posteriors are encoded as curved manifolds in representation space.

Linear interventions often move representations off-manifold, causing unintended effects.

Geometry-aware methods preserve the structure of belief manifolds during interventions.

Abstract

Large language models (LLMs) form implicit beliefs (posteriors over latent variables) from prompts, but we lack a mechanistic account of how these beliefs are encoded in representation space, how they update with new evidence, and how interventions reshape them. We study a controlled setting in which Llama-3.2 infers the parameters of a normal distribution from in-context samples. We show that parameter posteriors are encoded as curved manifolds in representation space, and trace how they evolve along the prompt. Standard linear steering moves representations off-manifold, inducing unintended, coupled changes, whereas geometry-aware methods preserve the target belief family. Our work demonstrates an example of linear field probing (LFP) as a principled approach to tile the data manifold and make interventions that respect the underlying geometry. Our results suggest that LLM beliefs are inherently geometric objects, and that globally linear representations are often inadequate abstractions.