SCALAR: Benchmarking SAE Interaction Sparsity in Toy LLMs

TL;DR

This paper introduces SCALAR, a benchmark for measuring interaction sparsity in sparse autoencoders within neural networks, and proposes Staircase SAEs to improve sparsity and interpretability across different model architectures.

Contribution

We present SCALAR, a novel benchmark for interaction sparsity, and propose Staircase SAEs with weight-sharing to enhance sparsity and interpretability in neural network features.

Findings

Staircase SAEs improve interaction sparsity by up to 63% over TopK SAEs.

JSAEs improve sparsity by 8.5% over TopK in feedforward layers but struggle with transformer blocks.

Staircase SAEs maintain sparsity benefits in both toy models and GPT-2 Small, preserving interpretability.

Abstract

Mechanistic interpretability aims to decompose neural networks into interpretable features and map their connecting circuits. The standard approach trains sparse autoencoders (SAEs) on each layer's activations. However, SAEs trained in isolation don't encourage sparse cross-layer connections, inflating extracted circuits where upstream features needlessly affect multiple downstream features. Current evaluations focus on individual SAE performance, leaving interaction sparsity unexamined. We introduce SCALAR (Sparse Connectivity Assessment of Latent Activation Relationships), a benchmark measuring interaction sparsity between SAE features. We also propose "Staircase SAEs", using weight-sharing to limit upstream feature duplication across downstream features. Using SCALAR, we compare TopK SAEs, Jacobian SAEs (JSAEs), and Staircase SAEs. Staircase SAEs improve relative sparsity over TopK SAEs by $59.67\% \pm 1.83\%$ (feedforward) and $63.15\% \pm 1.35\%$ (transformer blocks). JSAEs provide $8.54\% \pm 0.38\%$ improvement over TopK for feedforward layers but cannot train effectively across transformer blocks, unlike Staircase and TopK SAEs which work anywhere in the residual stream. We validate on a $216$K-parameter toy model and GPT-$2$ Small ($124$M), where Staircase SAEs maintain interaction sparsity improvements while preserving feature interpretability. Our work highlights the importance of interaction sparsity in SAEs through benchmarking and comparing promising architectures.