Explainability is NOT a Game

TL;DR

This paper critically examines the use of Shapley values in explainable AI, revealing that they can misrepresent feature importance by overvaluing irrelevant features and undervaluing relevant ones, thus questioning their reliability.

Contribution

It provides a simple yet powerful argument demonstrating the potential misleading nature of Shapley values in measuring feature importance in XAI.

Findings

Shapley values can assign importance to irrelevant features.

Shapley values can undervalue relevant features.

This challenges the reliability of many XAI methods using Shapley values.

Abstract

Explainable artificial intelligence (XAI) aims to help human decision-makers in understanding complex machine learning (ML) models. One of the hallmarks of XAI are measures of relative feature importance, which are theoretically justified through the use of Shapley values. This paper builds on recent work and offers a simple argument for why Shapley values can provide misleading measures of relative feature importance, by assigning more importance to features that are irrelevant for a prediction, and assigning less importance to features that are relevant for a prediction. The significance of these results is that they effectively challenge the many proposed uses of measures of relative feature importance in a fast-growing range of high-stakes application domains.