On Generating Plausible Counterfactual and Semi-Factual Explanations for Deep Learning

TL;DR

This paper introduces PIECE, a novel method for generating plausible counterfactual and semifactual explanations for deep learning image classifiers, improving explanation plausibility and aligning with human reasoning.

Contribution

The paper presents PIECE, a new approach for creating realistic counterfactual and semifactual explanations for CNNs in computer vision, addressing data manifold and feature normality issues.

Findings

PIECE produces more plausible counterfactuals than existing methods.

PIECE generates the most convincing semifactual explanations.

Controlled experiments validate the effectiveness of PIECE.

Abstract

There is a growing concern that the recent progress made in AI, especially regarding the predictive competence of deep learning models, will be undermined by a failure to properly explain their operation and outputs. In response to this disquiet counterfactual explanations have become massively popular in eXplainable AI (XAI) due to their proposed computational psychological, and legal benefits. In contrast however, semifactuals, which are a similar way humans commonly explain their reasoning, have surprisingly received no attention. Most counterfactual methods address tabular rather than image data, partly due to the nondiscrete nature of the latter making good counterfactuals difficult to define. Additionally generating plausible looking explanations which lie on the data manifold is another issue which hampers progress. This paper advances a novel method for generating plausible counterfactuals (and semifactuals) for black box CNN classifiers doing computer vision. The present method, called PlausIble Exceptionality-based Contrastive Explanations (PIECE), modifies all exceptional features in a test image to be normal from the perspective of the counterfactual class (hence concretely defining a counterfactual). Two controlled experiments compare this method to others in the literature, showing that PIECE not only generates the most plausible counterfactuals on several measures, but also the best semifactuals.