Revisiting The Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Experimental Analysis

TL;DR

This paper introduces a new set of metrics for evaluating Class Activation Mapping (CAM) explanations, improving reproducibility and comparison across methods through extensive experiments on ImageNet.

Contribution

It proposes novel metrics for CAM evaluation, enhancing effectiveness and reproducibility in explainability assessments.

Findings

New metrics outperform existing evaluation methods

Comprehensive comparison of CAM techniques on ImageNet

Improved reproducibility in CAM evaluation

Abstract

As the request for deep learning solutions increases, the need for explainability is even more fundamental. In this setting, particular attention has been given to visualization techniques, that try to attribute the right relevance to each input pixel with respect to the output of the network. In this paper, we focus on Class Activation Mapping (CAM) approaches, which provide an effective visualization by taking weighted averages of the activation maps. To enhance the evaluation and the reproducibility of such approaches, we propose a novel set of metrics to quantify explanation maps, which show better effectiveness and simplify comparisons between approaches. To evaluate the appropriateness of the proposal, we compare different CAM-based visualization methods on the entire ImageNet validation set, fostering proper comparisons and reproducibility.