SIDEs: Separating Idealization from Deceptive Explanations in xAI

TL;DR

This paper proposes a new framework called SIDEs for evaluating whether explainable AI methods use successful idealizations or deceptive distortions, addressing criticisms of current xAI techniques and promoting more trustworthy explanations.

Contribution

It introduces a novel framework for assessing xAI explanations based on idealization principles from science, distinguishing between successful idealizations and deception in model explanations.

Findings

Leading feature importance methods often involve idealization failure.

Counterfactual explanations can also be subject to idealization issues.

Remedies are suggested to improve the trustworthiness of xAI explanations.

Abstract

Explainable AI (xAI) methods are important for establishing trust in using black-box models. However, recent criticism has mounted against current xAI methods that they disagree, are necessarily false, and can be manipulated, which has started to undermine the deployment of black-box models. Rudin (2019) goes so far as to say that we should stop using black-box models altogether in high-stakes cases because xAI explanations "must be wrong". However, strict fidelity to the truth is historically not a desideratum in science. Idealizations -- the intentional distortions introduced to scientific theories and models -- are commonplace in the natural sciences and are seen as a successful scientific tool. Thus, it is not falsehood qua falsehood that is the issue. In this paper, I outline the need for xAI research to engage in idealization evaluation. Drawing on the use of idealizations in the natural sciences and philosophy of science, I introduce a novel framework for evaluating whether xAI methods engage in successful idealizations or deceptive explanations (SIDEs). SIDEs evaluates whether the limitations of xAI methods, and the distortions that they introduce, can be part of a successful idealization or are indeed deceptive distortions as critics suggest. I discuss the role that existing research can play in idealization evaluation and where innovation is necessary. Through a qualitative analysis we find that leading feature importance methods and counterfactual explanations are subject to idealization failure and suggest remedies for ameliorating idealization failure.