Interpreting CLIP with Hierarchical Sparse Autoencoders

TL;DR

This paper introduces Matryoshka SAE, a hierarchical autoencoder that improves interpretability of CLIP by balancing reconstruction quality and sparsity, enabling extraction of semantic concepts for analysis and control.

Contribution

The paper proposes MSAE, a hierarchical autoencoder that optimizes both sparsity and reconstruction quality simultaneously, advancing interpretability of large-scale vision-language models.

Findings

MSAE achieves state-of-the-art Pareto frontier between sparsity and reconstruction quality.

MSAE extracts over 120 semantic concepts from CLIP representations.

MSAE enables concept-based similarity search and bias analysis.

Abstract

Sparse autoencoders (SAEs) are useful for detecting and steering interpretable features in neural networks, with particular potential for understanding complex multimodal representations. Given their ability to uncover interpretable features, SAEs are particularly valuable for analyzing large-scale vision-language models (e.g., CLIP and SigLIP), which are fundamental building blocks in modern systems yet remain challenging to interpret and control. However, current SAE methods are limited by optimizing both reconstruction quality and sparsity simultaneously, as they rely on either activation suppression or rigid sparsity constraints. To this end, we introduce Matryoshka SAE (MSAE), a new architecture that learns hierarchical representations at multiple granularities simultaneously, enabling a direct optimization of both metrics without compromise. MSAE establishes a new state-of-the-art Pareto frontier between reconstruction quality and sparsity for CLIP, achieving 0.99 cosine similarity and less than 0.1 fraction of variance unexplained while maintaining ~80% sparsity. Finally, we demonstrate the utility of MSAE as a tool for interpreting and controlling CLIP by extracting over 120 semantic concepts from its representation to perform concept-based similarity search and bias analysis in downstream tasks like CelebA. We make the codebase available at https://github.com/WolodjaZ/MSAE.