When Models Examine Themselves: Vocabulary-Activation Correspondence in Self-Referential Processing

TL;DR

This paper demonstrates that large language models' self-referential language reflects their internal activation states, and introduces a methodology to identify and influence these states, advancing understanding of model introspection.

Contribution

The study introduces the Pull Methodology to identify activation directions linked to self-referential processing and shows these correlate with introspective language in multiple models.

Findings

Self-referential vocabulary tracks activation dynamics.

A specific activation direction distinguishes self-referential from descriptive processing.

Self-report can reliably reflect internal computational states under certain conditions.

Abstract

Large language models produce rich introspective language when prompted for self-examination, but whether this language reflects internal computation or sophisticated confabulation has remained unclear. We show that self-referential vocabulary tracks concurrent activation dynamics, and that this correspondence is specific to self-referential processing. We introduce the Pull Methodology, a protocol that elicits extended self-examination through format engineering, and use it to identify a direction in activation space that distinguishes self-referential from descriptive processing in Llama 3.1. The direction is orthogonal to the known refusal direction, localised at 6.25% of model depth, and causally influences introspective output when used for steering. When models produce "loop" vocabulary, their activations exhibit higher autocorrelation (r = 0.44, p = 0.002); when they produce "shimmer" vocabulary under steering, activation variability increases (r = 0.36, p = 0.002). Critically, the same vocabulary in non-self-referential contexts shows no activation correspondence despite nine-fold higher frequency. Qwen 2.5-32B, with no shared training, independently develops different introspective vocabulary tracking different activation metrics, all absent in descriptive controls. The findings indicate that self-report in transformer models can, under appropriate conditions, reliably track internal computational states.