Probing for Representation Manifolds in Superposition

TL;DR

This paper presents the Manifold Probe, a supervised technique to identify and analyze representation manifolds in superposition within language models, revealing interpretable features related to time and space.

Contribution

The Manifold Probe generalizes linear regression probes to discover and manipulate representation manifolds, demonstrating causal influence on model outputs.

Findings

Manifold Probe identifies interpretable time and space features in Llama 2-7b.

Steering along the time manifold affects model predictions about historical dates.

The method uncovers manifolds causally involved in model behavior.

Abstract

This paper introduces the Manifold Probe, a supervised method for discovering representation manifolds in superposition. The method generalizes linear regression probes by learning the space of features of a concept that can be linearly predicted from the representations, and then learning the directions used to encode them. We demonstrate the probe on representations of time and space in Llama 2-7b, finding manifolds which linearly represent an interpretable set of features in each case. In the case of time, we show that by steering along the manifold, we can influence the model's completions about the years in which famous songs, movies and books were released, providing evidence that the Manifold Probe can discover manifolds which are causally involved in model behaviour.