Suboptimal Shapley Value Explanations

TL;DR

This paper analyzes the limitations of current baseline choices in Shapley value explanations for DNNs, proposing an uncertainty-based reweighting method to improve explanation accuracy and consistency with human understanding.

Contribution

It identifies the problematic baseline causing bias in Shapley explanations and introduces a novel reweighting mechanism to enhance explanation quality and computational efficiency.

Findings

The proposed reweighting improves explanation consistency.

The method accelerates Shapley value computation.

Experiments validate the effectiveness across NLP tasks.

Abstract

Deep Neural Networks (DNNs) have demonstrated strong capacity in supporting a wide variety of applications. Shapley value has emerged as a prominent tool to analyze feature importance to help people understand the inference process of deep neural models. Computing Shapley value function requires choosing a baseline to represent feature's missingness. However, existing random and conditional baselines could negatively influence the explanation. In this paper, by analyzing the suboptimality of different baselines, we identify the problematic baseline where the asymmetric interaction between $\bm{x}'_i$ (the replacement of the faithful influential feature) and other features has significant directional bias toward the model's output, and conclude that $p(y|\bm{x}'_i) = p(y)$ potentially minimizes the asymmetric interaction involving $\bm{x}'_i$. We further generalize the uninformativeness of $\bm{x}'_i$ toward the label space $L$ to avoid estimating $p(y)$ and design a simple uncertainty-based reweighting mechanism to accelerate the computation process. We conduct experiments on various NLP tasks and our quantitative analysis demonstrates the effectiveness of the proposed uncertainty-based reweighting mechanism. Furthermore, by measuring the consistency of explanations generated by explainable methods and human, we highlight the disparity between model inference and human understanding.