Evaluating Sparse Autoencoders on Targeted Concept Erasure Tasks

TL;DR

This paper introduces automated evaluation metrics for Sparse Autoencoders (SAEs) using SHIFT and TPP, enabling better assessment of their interpretability and ability to disentangle concepts across models.

Contribution

The paper develops automated, scalable metrics SHIFT and TPP for evaluating SAE interpretability and concept disentanglement, replacing subjective human judgment.

Findings

Metrics effectively differentiate SAE training hyperparameters.

Metrics distinguish between different SAE architectures.

Automated evaluation correlates with interpretability quality.

Abstract

Sparse Autoencoders (SAEs) are an interpretability technique aimed at decomposing neural network activations into interpretable units. However, a major bottleneck for SAE development has been the lack of high-quality performance metrics, with prior work largely relying on unsupervised proxies. In this work, we introduce a family of evaluations based on SHIFT, a downstream task from Marks et al. (Sparse Feature Circuits, 2024) in which spurious cues are removed from a classifier by ablating SAE features judged to be task-irrelevant by a human annotator. We adapt SHIFT into an automated metric of SAE quality; this involves replacing the human annotator with an LLM. Additionally, we introduce the Targeted Probe Perturbation (TPP) metric that quantifies an SAE's ability to disentangle similar concepts, effectively scaling SHIFT to a wider range of datasets. We apply both SHIFT and TPP to multiple open-source models, demonstrating that these metrics effectively differentiate between various SAE training hyperparameters and architectures.