One Explanation is Not Enough: Structured Attention Graphs for Image Classification

TL;DR

This paper introduces structured attention graphs (SAGs) to represent multiple explanation maps for image classification, providing deeper insights into classifier decisions beyond single attention maps.

Contribution

The paper proposes a novel method to compute and visualize SAGs, capturing how different image regions collectively influence classification confidence.

Findings

Users are more accurate in answering counterfactual questions using SAGs.

SAGs provide a more comprehensive understanding of model decisions.

The approach enhances interpretability of convolutional networks.

Abstract

Attention maps are a popular way of explaining the decisions of convolutional networks for image classification. Typically, for each image of interest, a single attention map is produced, which assigns weights to pixels based on their importance to the classification. A single attention map, however, provides an incomplete understanding since there are often many other maps that explain a classification equally well. In this paper, we introduce structured attention graphs (SAGs), which compactly represent sets of attention maps for an image by capturing how different combinations of image regions impact a classifier's confidence. We propose an approach to compute SAGs and a visualization for SAGs so that deeper insight can be gained into a classifier's decisions. We conduct a user study comparing the use of SAGs to traditional attention maps for answering counterfactual questions about image classifications. Our results show that the users are more correct when answering comparative counterfactual questions based on SAGs compared to the baselines.