Comparing Baseline Shapley and Integrated Gradients for Local Explanation: Some Additional Insights

TL;DR

This paper compares two local explanation methods, Integrated Gradients and Baseline Shapley, analyzing their similarities and differences for tabular data and neural networks with ReLU activation.

Contribution

It provides additional insights into the comparative behavior of the two methods, including scenarios of agreement and divergence, supported by simulation studies.

Findings

Methods often yield different explanations for the same model.

Identifies situations where explanations from both methods are identical.

Simulation results highlight differences in neural network explanations.

Abstract

There are many different methods in the literature for local explanation of machine learning results. However, the methods differ in their approaches and often do not provide same explanations. In this paper, we consider two recent methods: Integrated Gradients (Sundararajan, Taly, & Yan, 2017) and Baseline Shapley (Sundararajan and Najmi, 2020). The original authors have already studied the axiomatic properties of the two methods and provided some comparisons. Our work provides some additional insights on their comparative behavior for tabular data. We discuss common situations where the two provide identical explanations and where they differ. We also use simulation studies to examine the differences when neural networks with ReLU activation function is used to fit the models.