Counterfactual Explanation Based on Gradual Construction for Deep Networks

TL;DR

This paper introduces a novel counterfactual explanation method for deep networks that gradually constructs explanations by iteratively selecting and optimizing features, resulting in clearer, more human-friendly interpretations aligned with training data distributions.

Contribution

The proposed method uniquely combines masking and composition steps to produce more realistic and understandable counterfactual explanations based on training data statistics.

Findings

Produces human-friendly interpretations across various datasets

Achieves explanations with fewer feature modifications

Verifies alignment with training data distribution

Abstract

To understand the black-box characteristics of deep networks, counterfactual explanation that deduces not only the important features of an input space but also how those features should be modified to classify input as a target class has gained an increasing interest. The patterns that deep networks have learned from a training dataset can be grasped by observing the feature variation among various classes. However, current approaches perform the feature modification to increase the classification probability for the target class irrespective of the internal characteristics of deep networks. This often leads to unclear explanations that deviate from real-world data distributions. To address this problem, we propose a counterfactual explanation method that exploits the statistics learned from a training dataset. Especially, we gradually construct an explanation by iterating over masking and composition steps. The masking step aims to select an important feature from the input data to be classified as a target class. Meanwhile, the composition step aims to optimize the previously selected feature by ensuring that its output score is close to the logit space of the training data that are classified as the target class. Experimental results show that our method produces human-friendly interpretations on various classification datasets and verify that such interpretations can be achieved with fewer feature modification.