A novel post-hoc explanation comparison metric and applications

TL;DR

This paper introduces the Shreyan Distance, a new metric for comparing explanation systems' feature importance rankings, and demonstrates its application in assessing consistency between SHAP and LIME across different learning tasks.

Contribution

The paper proposes the Shreyan Distance metric and provides the XAISuite library for integrating it into machine learning workflows, enhancing explanation comparison.

Findings

Shreyan Distance effectively quantifies differences between explanation systems.

Consistency between explainers varies with the type of learning task.

The XAISuite library facilitates practical application of the metric.

Abstract

Explanatory systems make the behavior of machine learning models more transparent, but are often inconsistent. To quantify the differences between explanatory systems, this paper presents the Shreyan Distance, a novel metric based on the weighted difference between ranked feature importance lists produced by such systems. This paper uses the Shreyan Distance to compare two explanatory systems, SHAP and LIME, for both regression and classification learning tasks. Because we find that the average Shreyan Distance varies significantly between these two tasks, we conclude that consistency between explainers not only depends on inherent properties of the explainers themselves, but also the type of learning task. This paper further contributes the XAISuite library, which integrates the Shreyan distance algorithm into machine learning pipelines.