A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributions

TL;DR

This paper critically examines the Integrated Gradients attribution method, clarifies its theoretical foundations, extends its applicability to internal neuron attributions, and introduces a new efficient method for identifying influential neurons.

Contribution

It clarifies the theoretical basis of IG, addresses its sensitivity properties, extends axioms to distribution-based baselines, and proposes a novel efficient neuron attribution method.

Findings

Identified key differences in function spaces affecting IG's uniqueness.

Established conditions under which IG is Lipschitz continuous.

Developed a new efficient method for internal neuron attribution.

Abstract

As deep learning (DL) efficacy grows, concerns for poor model explainability grow also. Attribution methods address the issue of explainability by quantifying the importance of an input feature for a model prediction. Among various methods, Integrated Gradients (IG) sets itself apart by claiming other methods failed to satisfy desirable axioms, while IG and methods like it uniquely satisfy said axioms. This paper comments on fundamental aspects of IG and its applications/extensions: 1) We identify key differences between IG function spaces and the supporting literature's function spaces which problematize previous claims of IG uniqueness. We show that with the introduction of an additional axiom, \textit{non-decreasing positivity}, the uniqueness claims can be established. 2) We address the question of input sensitivity by identifying function classes where IG is/is not Lipschitz in the attributed input. 3) We show that axioms for single-baseline methods have analogous properties for methods with probability distribution baselines. 4) We introduce a computationally efficient method of identifying internal neurons that contribute to specified regions of an IG attribution map. Finally, we present experimental results validating this method.