COSE: A Consistency-Sensitivity Metric for Saliency on Image Classification

TL;DR

This paper introduces COSE, a new metric for evaluating the reliability of saliency maps in image classification, emphasizing the balance between consistency and sensitivity to data augmentations.

Contribution

The paper proposes the COSE metric to assess saliency explanations' equivariance and invariance, revealing insights into model explanations across architectures.

Findings

Transformer models are better explained by saliency methods than CNNs.

GradCAM outperforms other methods in COSE but lacks variability on fine-grained datasets.

Balancing consistency and sensitivity is crucial for faithful saliency maps.

Abstract

We present a set of metrics that utilize vision priors to effectively assess the performance of saliency methods on image classification tasks. To understand behavior in deep learning models, many methods provide visual saliency maps emphasizing image regions that most contribute to a model prediction. However, there is limited work on analyzing the reliability of saliency methods in explaining model decisions. We propose the metric COnsistency-SEnsitivity (COSE) that quantifies the equivariant and invariant properties of visual model explanations using simple data augmentations. Through our metrics, we show that although saliency methods are thought to be architecture-independent, most methods could better explain transformer-based models over convolutional-based models. In addition, GradCAM was found to outperform other methods in terms of COSE but was shown to have limitations such as lack of variability for fine-grained datasets. The duality between consistency and sensitivity allow the analysis of saliency methods from different angles. Ultimately, we find that it is important to balance these two metrics for a saliency map to faithfully show model behavior.