Can Cross-Layer Transcoders Replace Vision Transformer Activations? An Interpretable Perspective on Vision

TL;DR

This paper introduces Cross-Layer Transcoders (CLTs) as interpretable, sparse proxy models for Vision Transformers, enabling layer-resolved understanding of representations while maintaining high fidelity and classification performance.

Contribution

The paper proposes CLTs as a novel, depth-aware proxy for ViT activations, providing faithful interpretability and improved understanding of layer contributions.

Findings

CLTs achieve high reconstruction fidelity of ViT activations.

CLTs preserve and sometimes improve CLIP zero-shot classification accuracy.

Cross-layer contribution scores reveal dominant layers critical for performance.

Abstract

Understanding the internal activations of Vision Transformers (ViTs) is critical for building interpretable and trustworthy models. While Sparse Autoencoders (SAEs) have been used to extract human-interpretable features, they operate on individual layers and fail to capture the cross-layer computational structure of Transformers, as well as the relative significance of each layer in forming the last-layer representation. Alternatively, we introduce the adoption of Cross-Layer Transcoders (CLTs) as reliable, sparse, and depth-aware proxy models for MLP blocks in ViTs. CLTs use an encoder-decoder scheme to reconstruct each post-MLP activation from learned sparse embeddings of preceding layers, yielding a linear decomposition that transforms the final representation of ViTs from an opaque embedding into an additive, layer-resolved construction that enables faithful attribution and process-level interpretability. We train CLTs on CLIP ViT-B/32 and ViT-B/16 across CIFAR-100, COCO, and ImageNet-100. We show that CLTs achieve high reconstruction fidelity with post-MLP activations while preserving and even improving, in some cases, CLIP zero-shot classification accuracy. In terms of interpretability, we show that the cross-layer contribution scores provide faithful attribution, revealing that the final representation is concentrated in a smaller set of dominant layer-wise terms whose removal degrades performance and whose retention largely preserves it. These results showcase the significance of adopting CLTs as an alternative interpretable proxy of ViTs in the vision domain.