An Imprecise SHAP as a Tool for Explaining the Class Probability Distributions under Limited Training Data

TL;DR

This paper introduces an imprecise SHAP method for explaining class probability distributions when training data is limited, using interval-valued Shapley values and linear optimization techniques.

Contribution

It proposes a novel approach to compute and reduce interval-valued Shapley values for imprecise probability distributions, extending SHAP to limited data scenarios.

Findings

Demonstrates effectiveness on synthetic data

Shows applicability to real-world datasets

Provides a computational framework using linear optimization

Abstract

One of the most popular methods of the machine learning prediction explanation is the SHapley Additive exPlanations method (SHAP). An imprecise SHAP as a modification of the original SHAP is proposed for cases when the class probability distributions are imprecise and represented by sets of distributions. The first idea behind the imprecise SHAP is a new approach for computing the marginal contribution of a feature, which fulfils the important efficiency property of Shapley values. The second idea is an attempt to consider a general approach to calculating and reducing interval-valued Shapley values, which is similar to the idea of reachable probability intervals in the imprecise probability theory. A simple special implementation of the general approach in the form of linear optimization problems is proposed, which is based on using the Kolmogorov-Smirnov distance and imprecise contamination models. Numerical examples with synthetic and real data illustrate the imprecise SHAP.