Feature Hedging: Correlated Features Break Narrow Sparse Autoencoders

TL;DR

This paper identifies a phenomenon called feature hedging in narrow sparse autoencoders, where correlated features merge, reducing interpretability, especially in large language models, and proposes an improved SAE variant to mitigate this issue.

Contribution

It introduces the concept of feature hedging caused by SAE reconstruction loss, analyzes it theoretically and empirically, and proposes a new SAE variant to address the problem.

Findings

Feature hedging causes correlated features to merge in narrow SAEs.

Narrower SAEs are more susceptible to feature hedging.

The proposed matryoshka SAE variant reduces feature hedging effects.

Abstract

It is assumed that sparse autoencoders (SAEs) decompose polysemantic activations into interpretable linear directions, as long as the activations are composed of sparse linear combinations of underlying features. However, we find that if an SAE is more narrow than the number of underlying "true features" on which it is trained, and there is correlation between features, the SAE will merge components of correlated features together, thus destroying monosemanticity. In LLM SAEs, these two conditions are almost certainly true. This phenomenon, which we call feature hedging, is caused by SAE reconstruction loss, and is more severe the narrower the SAE. In this work, we introduce the problem of feature hedging and study it both theoretically in toy models and empirically in SAEs trained on LLMs. We suspect that feature hedging may be one of the core reasons that SAEs consistently underperform supervised baselines. Finally, we use our understanding of feature hedging to propose an improved variant of matryoshka SAEs. Importantly, our work shows that SAE width is not a neutral hyperparameter: narrower SAEs suffer more from hedging than wider SAEs.