Inside-Out: Measuring Generalization in Vision Transformers Through Inner Workings

TL;DR

This paper introduces two novel, circuit-based metrics for evaluating vision transformer generalization, outperforming existing proxies in predicting performance before and after deployment under distribution shifts.

Contribution

It proposes a new approach using internal circuit mechanisms of models as reliable, label-free proxies for generalization performance in vision transformers.

Findings

Dependency Depth Bias correlates with model generalization on target data.

Circuit Shift Score predicts model performance under distribution shifts.

Both metrics outperform existing proxies by over 13% and 34%.

Abstract

Reliable generalization metrics are fundamental to the evaluation of machine learning models. Especially in high-stakes applications where labeled target data are scarce, evaluation of models' generalization performance under distribution shift is a pressing need. We focus on two practical scenarios: (1) Before deployment, how to select the best model for unlabeled target data? (2) After deployment, how to monitor model performance under distribution shift? The central need in both cases is a reliable and label-free proxy metric. Yet existing proxy metrics, such as model confidence or accuracy-on-the-line, are often unreliable as they only assess model output while ignoring the internal mechanisms that produce them. We address this limitation by introducing a new perspective: using the inner workings of a model, i.e., circuits, as a predictive metric of generalization performance. Leveraging circuit discovery, we extract the causal interactions between internal representations as a circuit, from which we derive two metrics tailored to the two practical scenarios. (1) Before deployment, we introduce Dependency Depth Bias, which measures different models' generalization capability on target data. (2) After deployment, we propose Circuit Shift Score, which predicts a model's generalization under different distribution shifts. Across various tasks, both metrics demonstrate significantly improved correlation with generalization performance, outperforming existing proxies by an average of 13.4\% and 34.1\%, respectively. Our code is available at https://github.com/deep-real/GenCircuit.