A-FMI: Learning Attributions from Deep Networks via Feature Map Importance

TL;DR

A-FMI is a novel attribution method that improves CNN interpretability by addressing gradient saturation and feature redundancy through feature map importance and reference coupling, demonstrating superior performance on ImageNet.

Contribution

The paper introduces feature map importance (FMI) and the A-FMI attribution method, which enhances explanation faithfulness and robustness in CNNs by coupling target and reference images.

Findings

A-FMI outperforms popular attribution methods in faithfulness.

A-FMI is insensitive to reference choice.

A-FMI improves class discriminability and explanation quality.

Abstract

Gradient-based attribution methods can aid in the understanding of convolutional neural networks (CNNs). However, the redundancy of attribution features and the gradient saturation problem, which weaken the ability to identify significant features and cause an explanation focus shift, are challenges that attribution methods still face. In this work, we propose: 1) an essential characteristic, Strong Relevance, when selecting attribution features; 2) a new concept, feature map importance (FMI), to refine the contribution of each feature map, which is faithful to the CNN model; and 3) a novel attribution method via FMI, termed A-FMI, to address the gradient saturation problem, which couples the target image with a reference image, and assigns the FMI to the difference-from-reference at the granularity of feature map. Through visual inspections and qualitative evaluations on the ImageNet dataset, we show the compelling advantages of A-FMI on its faithfulness, insensitivity to the choice of reference, class discriminability, and superior explanation performance compared with popular attribution methods across varying CNN architectures.