XAI Handbook: Towards a Unified Framework for Explainable AI

TL;DR

This paper introduces a unified theoretical framework for explainable AI that standardizes terminology, guides explanation generation, and enables comparison of different methods like LIME, SHAP, and MDNet.

Contribution

It provides concrete definitions for key XAI terms and a comprehensive framework to evaluate and compare existing explanation methods.

Findings

Framework aligns with explanation and interpretability desiderata

Enables comparison of LIME, SHAP, and MDNet

Highlights advantages and shortcomings of methods

Abstract

The field of explainable AI (XAI) has quickly become a thriving and prolific community. However, a silent, recurrent and acknowledged issue in this area is the lack of consensus regarding its terminology. In particular, each new contribution seems to rely on its own (and often intuitive) version of terms like "explanation" and "interpretation". Such disarray encumbers the consolidation of advances in the field towards the fulfillment of scientific and regulatory demands e.g., when comparing methods or establishing their compliance with respect to biases and fairness constraints. We propose a theoretical framework that not only provides concrete definitions for these terms, but it also outlines all steps necessary to produce explanations and interpretations. The framework also allows for existing contributions to be re-contextualized such that their scope can be measured, thus making them comparable to other methods. We show that this framework is compliant with desiderata on explanations, on interpretability and on evaluation metrics. We present a use-case showing how the framework can be used to compare LIME, SHAP and MDNet, establishing their advantages and shortcomings. Finally, we discuss relevant trends in XAI as well as recommendations for future work, all from the standpoint of our framework.