Conditional expectation network for SHAP

TL;DR

This paper introduces a neural network-based method to efficiently compute the conditional SHAP values, accounting for feature dependence, applicable to neural networks and complex regression models, enhancing interpretability tools.

Contribution

It proposes a surrogate neural network approach for calculating conditional SHAP values that properly considers feature dependence, improving interpretability in complex models.

Findings

Efficient computation of conditional SHAP values for neural networks.

Enables dependence-aware drop1 and ANOVA analyses.

Provides a partial dependence plot that accounts for feature dependence.

Abstract

A very popular model-agnostic technique for explaining predictive models is the SHapley Additive exPlanation (SHAP). The two most popular versions of SHAP are a conditional expectation version and an unconditional expectation version (the latter is also known as interventional SHAP). Except for tree-based methods, usually the unconditional version is used (for computational reasons). We provide a (surrogate) neural network approach which allows us to efficiently calculate the conditional version for both neural networks and other regression models, and which properly considers the dependence structure in the feature components. This proposal is also useful to provide drop1 and anova analyses in complex regression models which are similar to their generalized linear model (GLM) counterparts, and we provide a partial dependence plot (PDP) counterpart that considers the right dependence structure in the feature components.