Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?

TL;DR

This paper investigates whether baseline masking accurately captures the absence of input variables in DNNs for Shapley value computation, proposing causal pattern analysis and a method to learn optimal baselines.

Contribution

It introduces a causal pattern-based framework to evaluate baseline faithfulness and proposes a method to learn optimal baseline values for Shapley value explanations.

Findings

Causal patterns can explain Shapley values.

Optimal baselines improve attribution fidelity.

Experimental results validate the proposed method.

Abstract

Masking some input variables of a deep neural network (DNN) and computing output changes on the masked input sample represent a typical way to compute attributions of input variables in the sample. People usually mask an input variable using its baseline value. However, there is no theory to examine whether baseline value faithfully represents the absence of an input variable, \emph{i.e.,} removing all signals from the input variable. Fortunately, recent studies show that the inference score of a DNN can be strictly disentangled into a set of causal patterns (or concepts) encoded by the DNN. Therefore, we propose to use causal patterns to examine the faithfulness of baseline values. More crucially, it is proven that causal patterns can be explained as the elementary rationale of the Shapley value. Furthermore, we propose a method to learn optimal baseline values, and experimental results have demonstrated its effectiveness.