Sparsification and Reconstruction from the Perspective of Representation Geometry

TL;DR

This paper explores how sparse autoencoders organize language model representations through geometric analysis, revealing their impact on feature disentanglement and reconstruction, and providing insights for interpretability tools.

Contribution

It introduces the SAEMA framework to analyze the geometric structure of sparse representations and links representational separability to reconstruction performance.

Findings

Stratified structure of representations validated by rank variability of SSPSD matrices.

Sparse encoding amplifies inter-feature distinctions and increases dimensionality.

Separable global representations causally improve reconstruction quality.

Abstract

Sparse Autoencoders (SAEs) have emerged as a predominant tool in mechanistic interpretability, aiming to identify interpretable monosemantic features. However, how does sparse encoding organize the representations of activation vector from language models? What is the relationship between this organizational paradigm and feature disentanglement as well as reconstruction performance? To address these questions, we propose the SAEMA, which validates the stratified structure of the representation by observing the variability of the rank of the symmetric semipositive definite (SSPD) matrix corresponding to the modal tensor unfolded along the latent tensor with the level of noise added to the residual stream. To systematically investigate how sparse encoding alters representational structures, we define local and global representations, demonstrating that they amplify inter-feature distinctions by merging similar semantic features and introducing additional dimensionality. Furthermore, we intervene the global representation from an optimization perspective, proving a significant causal relationship between their separability and the reconstruction performance. This study explains the principles of sparsity from the perspective of representational geometry and demonstrates the impact of changes in representational structure on reconstruction performance. Particularly emphasizes the necessity of understanding representations and incorporating representational constraints, providing empirical references for developing new interpretable tools and improving SAEs. The code is available at \hyperlink{https://github.com/wenjie1835/SAERepGeo}{https://github.com/wenjie1835/SAERepGeo}.