Aggregating Local Saliency Maps for Semi-Global Explainable Image Classification

TL;DR

This paper introduces Segment Attribution Tables (SATs), a novel method that summarises local saliency explanations into semi-global insights, helping to understand and debug image classifiers by highlighting influential image segments.

Contribution

The paper proposes SATs, a new approach to aggregate local saliency maps into semi-global explanations, bridging the gap between local and global interpretability in image classification.

Findings

SATs reveal model reliance on background and spurious correlations.

SATs can explain any classifier with saliency maps using segmentation.

SATs provide practical insights for debugging image classifiers.

Abstract

Deep learning dominates image classification tasks, yet understanding how models arrive at predictions remains a challenge. Much research focuses on local explanations of individual predictions, such as saliency maps, which visualise the influence of specific pixels on a model's prediction. However, reviewing many of these explanations to identify recurring patterns is infeasible, while global methods often oversimplify and miss important local behaviours. To address this, we propose Segment Attribution Tables (SATs), a method for summarising local saliency explanations into (semi-)global insights. SATs take image segments (such as "eyes" in Chihuahuas) and leverage saliency maps to quantify their influence. These segments highlight concepts the model relies on across instances and reveal spurious correlations, such as reliance on backgrounds or watermarks, even when out-of-distribution test performance sees little change. SATs can explain any classifier for which a form of saliency map can be produced, using segmentation maps that provide named segments. SATs bridge the gap between oversimplified global summaries and overly detailed local explanations, offering a practical tool for analysing and debugging image classifiers.