The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations

TL;DR

This paper advocates shifting from post-hoc explanations to designing inherently interpretable neural networks for human-centric XAI, emphasizing real-time, accurate, and consistent interpretability in AI systems.

Contribution

It introduces a paradigm change by proposing interpretable neural network architectures and two specific workflows to enhance human-centric explainability.

Findings

Identified five key needs for human-centric XAI.

Proposed two schemes: adaptive routing with InterpretCC and temporal diagnostics with I2MD.

Argued that intrinsically interpretable neural networks are the future of XAI.

Abstract

Explainable Artificial Intelligence (XAI) plays a crucial role in enabling human understanding and trust in deep learning systems. As models get larger, more ubiquitous, and pervasive in aspects of daily life, explainability is necessary to minimize adverse effects of model mistakes. Unfortunately, current approaches in human-centric XAI (e.g. predictive tasks in healthcare, education, or personalized ads) tend to rely on a single post-hoc explainer, whereas recent work has identified systematic disagreement between post-hoc explainers when applied to the same instances of underlying black-box models. In this paper, we therefore present a call for action to address the limitations of current state-of-the-art explainers. We propose a shift from post-hoc explainability to designing interpretable neural network architectures. We identify five needs of human-centric XAI (real-time, accurate, actionable, human-interpretable, and consistent) and propose two schemes for interpretable-by-design neural network workflows (adaptive routing with InterpretCC and temporal diagnostics with I2MD). We postulate that the future of human-centric XAI is neither in explaining black-boxes nor in reverting to traditional, interpretable models, but in neural networks that are intrinsically interpretable.