Formal Abductive Latent Explanations for Prototype-Based Networks

TL;DR

This paper introduces Abductive Latent Explanations (ALEs), a formal method to generate more reliable and formalized explanations for prototype-based neural networks, improving interpretability especially in safety-critical applications.

Contribution

It proposes ALEs, a novel formalism for explanations in prototype-based networks, combining interpretability with formal guarantees and offering scalable algorithms for diverse datasets.

Findings

ALEs improve explanation reliability in prototype-based models

The proposed algorithms are scalable and effective on image classification tasks

ALEs provide formal guarantees on the sufficiency of explanations

Abstract

Case-based reasoning networks are machine-learning models that make predictions based on similarity between the input and prototypical parts of training samples, called prototypes. Such models are able to explain each decision by pointing to the prototypes that contributed the most to the final outcome. As the explanation is a core part of the prediction, they are often qualified as ``interpretable by design". While promising, we show that such explanations are sometimes misleading, which hampers their usefulness in safety-critical contexts. In particular, several instances may lead to different predictions and yet have the same explanation. Drawing inspiration from the field of formal eXplainable AI (FXAI), we propose Abductive Latent Explanations (ALEs), a formalism to express sufficient conditions on the intermediate (latent) representation of the instance that imply the prediction. Our approach combines the inherent interpretability of case-based reasoning models and the guarantees provided by formal XAI. We propose a solver-free and scalable algorithm for generating ALEs based on three distinct paradigms, compare them, and present the feasibility of our approach on diverse datasets for both standard and fine-grained image classification. The associated code can be found at https://github.com/julsoria/ale