Class-Discriminative Attention Maps for Vision Transformers

TL;DR

This paper introduces class-discriminative attention maps (CDAM), a gradient-based method for explaining vision transformers by highlighting features relevant to specific classes or concepts, improving interpretability and relevance of explanations.

Contribution

The paper proposes CDAM, a novel gradient-based extension of attention maps for vision transformers that enhances class sensitivity and semantic relevance of explanations.

Findings

CDAM outperforms 7 other importance estimators on correctness, compactness, and class sensitivity.

Smooth and Integrated CDAM variants further improve explanation quality.

CDAM effectively explains medical image models for lung CT scans.

Abstract

Importance estimators are explainability methods that quantify feature importance for deep neural networks (DNN). In vision transformers (ViT), the self-attention mechanism naturally leads to attention maps, which are sometimes interpreted as importance scores that indicate which input features ViT models are focusing on. However, attention maps do not account for signals from downstream tasks. To generate explanations that are sensitive to downstream tasks, we have developed class-discriminative attention maps (CDAM), a gradient-based extension that estimates feature importance with respect to a known class or a latent concept. CDAM scales attention scores by how relevant the corresponding tokens are for the predictions of a classifier head. In addition to targeting the supervised classifier, CDAM can explain an arbitrary concept shared by selected samples by measuring similarity in the latent space of ViT. Additionally, we introduce Smooth CDAM and Integrated CDAM, which average a series of CDAMs with slightly altered tokens. Our quantitative benchmarks include correctness, compactness, and class sensitivity, in comparison to 7 other importance estimators. Vanilla, Smooth, and Integrated CDAM excel across all three benchmarks. In particular, our results suggest that existing importance estimators may not provide sufficient class-sensitivity. We demonstrate the utility of CDAM in medical images by training and explaining malignancy and biomarker prediction models based on lung Computed Tomography (CT) scans. Overall, CDAM is shown to be highly class-discriminative and semantically relevant, while providing compact explanations.