Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairs

TL;DR

Training lightweight adapters on interpretability artifacts enables reliable and scalable self-interpretation in language models without altering their core parameters, outperforming baseline methods across multiple tasks.

Contribution

Introducing a method to train minimal adapters on interpretability artifacts that significantly improve self-interpretation reliability and generalization without changing the underlying language model.

Findings

Adapters outperform training labels in autoencoder tasks (71% vs 63%)

High recall in topic identification (94%)

Implicit reasoning surfaces without chain-of-thought

Abstract

Self-interpretation methods prompt language models to describe their own internal states, but remain unreliable due to hyperparameter sensitivity. We show that training lightweight adapters on interpretability artifacts, while keeping the LM entirely frozen, yields reliable self-interpretation across tasks and model families. A scalar affine adapter with just $d_\text{model}+1$ parameters suffices: trained adapters generate sparse autoencoder feature labels that outperform the training labels themselves (71% vs 63% generation scoring at 70B scale), identify topics with 94% recall@1 versus 1% for untrained baselines, and decode bridge entities in multi-hop reasoning that appear in neither prompt nor response, surfacing implicit reasoning without chain-of-thought. The learned bias vector alone accounts for 85% of improvement, and simpler adapters generalize better than more expressive alternatives. Controlling for model knowledge via prompted descriptions, we find self-interpretation gains outpace capability gains from 7B to 72B parameters. Our results demonstrate that self-interpretation improves with scale, without modifying the model being interpreted.