Reproducibility review of "Why Not Other Classes": Towards Class-Contrastive Back-Propagation Explanations

TL;DR

This paper reproduces and extends a method for contrastive explanations in neural networks, evaluating its generalization across different models and addressing reproducibility issues with detailed analysis and open-source code.

Contribution

It reproduces the original contrastive explanation method, evaluates its generalization to other explanation techniques and models, and improves reproducibility by clarifying methodology and providing code.

Findings

Reproduced original results with similar heatmaps.

Method generalizes well to Vision Transformers and other back-propagation methods.

Identified issues in the original paper's methodology and equations.

Abstract

"Why Not Other Classes?": Towards Class-Contrastive Back-Propagation Explanations (Wang & Wang, 2022) provides a method for contrastively explaining why a certain class in a neural network image classifier is chosen above others. This method consists of using back-propagation-based explanation methods from after the softmax layer rather than before. Our work consists of reproducing the work in the original paper. We also provide extensions to the paper by evaluating the method on XGradCAM, FullGrad, and Vision Transformers to evaluate its generalization capabilities. The reproductions show similar results as the original paper, with the only difference being the visualization of heatmaps which could not be reproduced to look similar. The generalization seems to be generally good, with implementations working for Vision Transformers and alternative back-propagation methods. We also show that the original paper suffers from issues such as a lack of detail in the method and an erroneous equation which makes reproducibility difficult. To remedy this we provide an open-source repository containing all code used for this project.