Strengthening Interpretability: An Investigative Study of Integrated Gradient Methods

TL;DR

This paper evaluates the IDGI framework's effectiveness in improving Integrated Gradients explanations, analyzing its theoretical properties, numerical stability, and performance across different step counts in a comprehensive reproducibility study.

Contribution

It provides the first rigorous analysis and extensive experimental validation of the IDGI framework, enhancing the reliability of IG-based interpretability methods.

Findings

IDGI reduces explanation noise in IG methods

IDGI outperforms standard IG in key metrics

Numerical stability varies with step count

Abstract

We conducted a reproducibility study on Integrated Gradients (IG) based methods and the Important Direction Gradient Integration (IDGI) framework. IDGI eliminates the explanation noise in each step of the computation of IG-based methods that use the Riemann Integration for integrated gradient computation. We perform a rigorous theoretical analysis of IDGI and raise a few critical questions that we later address through our study. We also experimentally verify the authors' claims concerning the performance of IDGI over IG-based methods. Additionally, we varied the number of steps used in the Riemann approximation, an essential parameter in all IG methods, and analyzed the corresponding change in results. We also studied the numerical instability of the attribution methods to check the consistency of the saliency maps produced. We developed the complete code to implement IDGI over the baseline IG methods and evaluated them using three metrics since the available code was insufficient for this study.