Part-based Quantitative Analysis for Heatmaps

TL;DR

This paper emphasizes the importance of developing automatic, scalable, and quantitative methods for analyzing heatmaps in Explainable AI to improve objectivity, accessibility, and evaluation metrics.

Contribution

It highlights the need for novel quantitative analysis techniques and comprehensive metrics to evaluate heatmap quality at a granular level.

Findings

Current heatmap analysis is subjective and limited to experts.

Automatic and scalable analysis methods are needed for broader applicability.

Evaluation metrics for heatmap quality are underdeveloped.

Abstract

Heatmaps have been instrumental in helping understand deep network decisions, and are a common approach for Explainable AI (XAI). While significant progress has been made in enhancing the informativeness and accessibility of heatmaps, heatmap analysis is typically very subjective and limited to domain experts. As such, developing automatic, scalable, and numerical analysis methods to make heatmap-based XAI more objective, end-user friendly, and cost-effective is vital. In addition, there is a need for comprehensive evaluation metrics to assess heatmap quality at a granular level.