Iterative and Adaptive Sampling with Spatial Attention for Black-Box Model Explanations

TL;DR

This paper introduces IASSA, a novel framework for explaining deep neural network decisions by generating importance maps through iterative, adaptive sampling and long-range spatial attention, outperforming existing methods.

Contribution

The paper proposes IASSA, a new explainability method that uses affinity matrices and spatial attention for more accurate black-box model explanations.

Findings

Outperforms state-of-the-art explanation methods on MS-COCO

Effectively captures long-range pixel correlations

Provides more interpretable importance maps

Abstract

Deep neural networks have achieved great success in many real-world applications, yet it remains unclear and difficult to explain their decision-making process to an end-user. In this paper, we address the explainable AI problem for deep neural networks with our proposed framework, named IASSA, which generates an importance map indicating how salient each pixel is for the model's prediction with an iterative and adaptive sampling module. We employ an affinity matrix calculated on multi-level deep learning features to explore long-range pixel-to-pixel correlation, which can shift the saliency values guided by our long-range and parameter-free spatial attention. Extensive experiments on the MS-COCO dataset show that our proposed approach matches or exceeds the performance of state-of-the-art black-box explanation methods.