Benchmarking the Attribution Quality of Vision Models

TL;DR

This paper introduces a new evaluation protocol for attribution maps in vision models, enabling better comparison of methods and understanding of how model design influences attribution quality.

Contribution

It proposes a novel evaluation protocol addressing key limitations of existing methods and assesses how different vision model designs impact attribution quality.

Findings

Explainable models outperform standard models in attribution quality.

Raw attribution values have higher quality than previously reported.

Model design choices significantly influence attribution effectiveness.

Abstract

Attribution maps are one of the most established tools to explain the functioning of computer vision models. They assign importance scores to input features, indicating how relevant each feature is for the prediction of a deep neural network. While much research has gone into proposing new attribution methods, their proper evaluation remains a difficult challenge. In this work, we propose a novel evaluation protocol that overcomes two fundamental limitations of the widely used incremental-deletion protocol, i.e., the out-of-domain issue and lacking inter-model comparisons. This allows us to evaluate 23 attribution methods and how different design choices of popular vision backbones affect their attribution quality. We find that intrinsically explainable models outperform standard models and that raw attribution values exhibit a higher attribution quality than what is known from previous work. Further, we show consistent changes in the attribution quality when varying the network design, indicating that some standard design choices promote attribution quality.