The Rate-Distortion-Polysemanticity Tradeoff in SAEs

TL;DR

This paper explores the inherent tradeoff in Sparse Autoencoders between accuracy, efficiency, and interpretability, revealing how data distribution influences polysemanticity and proposing benchmarks for measuring it.

Contribution

It introduces the Rate-Distortion-Polysemanticity tradeoff in SAEs, combining theoretical analysis and empirical validation, and benchmarks proxy metrics on language models.

Findings

Restricting SAEs to be monosemantic increases rate and distortion.

Polysemanticity depends on feature co-occurrence probabilities in data.

Benchmarking proxy metrics on Large Language Models reveals data-driven polysemanticity influences.

Abstract

Sparse Autoencoders (SAEs) that can accurately reconstruct their input (minimizing distortion) by making efficient use of few features (minimizing the rate) often fail to learn monosemantic representations (highly interpretable), limiting their usefulness for mechanistic interpretability. In this paper, we characterise this tension in learning faithful, efficient, and interpretable explanations, introducing the Rate-Distortion-Polysemanticity tradeoff in SAEs. Under toy-modeling assumptions, we theoretically and empirically show that restricting the SAE to be monosemantic necessarily comes with an increase in rate and distortion. Assuming a generative model behind the input observations, we further demonstrate that the degree of polysemanticity of optimal SAEs is determined by the training data distribution, especially by the probability of features to co-occur. Finally, we extend the analysis to real-world settings by deriving necessary conditions that a polysemanticity measure should satisfy when the data-generating process is unknown, and we benchmark existing proxy metrics on SAEs trained on Large Language Models. Taken together, our findings show that polysemanticity is a data problem that should be accounted for when addressing it at the architectural and optimization level.