IDGI: A Framework to Eliminate Explanation Noise from Integrated Gradients

TL;DR

IDGI is a new framework that significantly reduces explanation noise in Integrated Gradients, enhancing interpretability of deep neural network decisions by analytically identifying and eliminating noise sources.

Contribution

The paper introduces IDGI, a novel framework that can be integrated into existing IG methods to minimize explanation noise based on analytical insights.

Findings

IDGI improves interpretability metrics across multiple IG-based methods.

Extensive experiments demonstrate significant noise reduction.

IDGI enhances the clarity of neural network explanations.

Abstract

Integrated Gradients (IG) as well as its variants are well-known techniques for interpreting the decisions of deep neural networks. While IG-based approaches attain state-of-the-art performance, they often integrate noise into their explanation saliency maps, which reduce their interpretability. To minimize the noise, we examine the source of the noise analytically and propose a new approach to reduce the explanation noise based on our analytical findings. We propose the Important Direction Gradient Integration (IDGI) framework, which can be easily incorporated into any IG-based method that uses the Reimann Integration for integrated gradient computation. Extensive experiments with three IG-based methods show that IDGI improves them drastically on numerous interpretability metrics.