Identifying Intervenable and Interpretable Features via Orthogonality Regularization

TL;DR

This paper introduces an orthogonality regularization technique to disentangle features in language models, enhancing interpretability and causal intervention capabilities without sacrificing performance.

Contribution

It proposes a novel orthogonality penalty that yields identifiable, modular features suitable for causal intervention in language models.

Findings

Orthogonality regularization improves feature interpretability.

Disentangled features enable isolated causal interventions.

Performance on target datasets remains stable with orthogonality penalty.

Abstract

With recent progress on fine-tuning language models around a fixed sparse autoencoder, we disentangle the decoder matrix into almost orthogonal features. This reduces interference and superposition between the features, while keeping performance on the target dataset essentially unchanged. Our orthogonality penalty leads to identifiable features, ensuring the uniqueness of the decomposition. Further, we find that the distance between embedded feature explanations increases with stricter orthogonality penalty, a desirable property for interpretability. Invoking the $\textit{Independent Causal Mechanisms}$ principle, we argue that orthogonality promotes modular representations amenable to causal intervention. We empirically show that these increasingly orthogonalized features allow for isolated interventions. Our code is available under $\texttt{https://github.com/mrtzmllr/sae-icm}$.