Pre or Post-Softmax Scores in Gradient-based Attribution Methods, What is Best?

TL;DR

This paper compares the effectiveness of using pre-softmax versus post-softmax scores in gradient-based attribution methods for neural network classifiers, highlighting practical differences and implications.

Contribution

It provides a detailed analysis of the advantages and disadvantages of pre-softmax and post-softmax gradients in attribution methods.

Findings

Pre-softmax gradients often provide clearer attribution signals.

Post-softmax gradients can be more stable but less informative.

The choice impacts the interpretability of attribution results.

Abstract

Gradient based attribution methods for neural networks working as classifiers use gradients of network scores. Here we discuss the practical differences between using gradients of pre-softmax scores versus post-softmax scores, and their respective advantages and disadvantages.