Evaluating counterfactual explanations using Pearl's counterfactual method

TL;DR

This paper introduces a method to evaluate counterfactual explanations by applying Judea Pearl's causal approach, revealing that many existing CEs conflict with true causal structures, emphasizing the importance of causal understanding.

Contribution

It is the first to apply Pearl's counterfactual method to evaluate machine learning-generated explanations, highlighting the impact of causal structure on explanation validity.

Findings

30% of CEs conflicted with Pearl's method

Causal structure significantly affects counterfactual explanations

Highlights need for causal awareness in explanation generation

Abstract

Counterfactual explanations (CEs) are methods for generating an alternative scenario that produces a different desirable outcome. For example, if a student is predicted to fail a course, then counterfactual explanations can provide the student with alternate ways so that they would be predicted to pass. The applications are many. However, CEs are currently generated from machine learning models that do not necessarily take into account the true causal structure in the data. By doing this, bias can be introduced into the CE quantities. I propose in this study to test the CEs using Judea Pearl's method of computing counterfactuals which has thus far, surprisingly, not been seen in the counterfactual explanation (CE) literature. I furthermore evaluate these CEs on three different causal structures to show how the true underlying causal structure affects the CEs that are generated. This study presented a method of evaluating CEs using Pearl's method and it showed, (although using a limited sample size), that thirty percent of the CEs conflicted with those computed by Pearl's method. This shows that we cannot simply trust CEs and it is vital for us to know the true causal structure before we blindly compute counterfactuals using the original machine learning model.