Zero-Ablation Overstates Register Content Dependence in DINO Vision Transformers

TL;DR

Zero-ablation overstates the importance of exact register content in vision transformers; alternative replacements show performance is maintained by plausible register-like activations.

Contribution

The paper demonstrates that zero-ablation overstates register dependence in vision transformers and introduces replacement controls that better reflect true functional importance.

Findings

Replacement controls preserve performance despite perturbations.

Zero-ablation causes disproportionately large internal representation perturbations.

Register-like activations, not exact values, underpin performance in frozen-feature evaluations.

Abstract

Zero-ablation -- replacing token activations with zero vectors -- is widely used to probe token function in vision transformers. Register zeroing in DINOv2+registers and DINOv3 produces large drops (up to $-36.6$\,pp classification, $-30.9$\,pp segmentation), suggesting registers are functionally indispensable. However, three replacement controls -- mean-substitution, noise-substitution, and cross-image register-shuffling -- preserve performance across classification, correspondence, and segmentation, remaining within ${\sim}1$\,pp of the unmodified baseline. Per-patch cosine similarity shows these replacements genuinely perturb internal representations, while zeroing causes disproportionately large perturbations, consistent with why it alone degrades tasks. We conclude that zero-ablation overstates dependence on exact register content. In the frozen-feature evaluations we test, performance depends on plausible register-like activations rather than on exact image-specific values. Registers nevertheless buffer dense features from \texttt{[CLS]} dependence and are associated with compressed patch geometry. These findings, including the replacement-control results, replicate at ViT-B scale.