Gradient-Based Interpretability Methods and Binarized Neural Networks

TL;DR

This paper evaluates the effectiveness of popular saliency map interpretability methods on Binarized Neural Networks (BNNs) versus full precision networks, revealing differences in explanation quality and consistency.

Contribution

It provides the first comparative analysis of saliency map methods on BNNs, highlighting their varying performance and the need for broader testing of interpretability techniques.

Findings

Gradient produces similar maps for BNNs and FPNNs

SmoothGrad yields noisier maps for BNNs

GradCAM explanations differ significantly between network types

Abstract

Binarized Neural Networks (BNNs) have the potential to revolutionize the way that deep learning is carried out in edge computing platforms. However, the effectiveness of interpretability methods on these networks has not been assessed. In this paper, we compare the performance of several widely used saliency map-based interpretabilty techniques (Gradient, SmoothGrad and GradCAM), when applied to Binarized or Full Precision Neural Networks (FPNNs). We found that the basic Gradient method produces very similar-looking maps for both types of network. However, SmoothGrad produces significantly noisier maps for BNNs. GradCAM also produces saliency maps which differ between network types, with some of the BNNs having seemingly nonsensical explanations. We comment on possible reasons for these differences in explanations and present it as an example of why interpretability techniques should be tested on a wider range of network types.