What Does Explainable AI Really Mean? A New Conceptualization of Perspectives

TL;DR

This paper redefines explainable AI by proposing three core notions—opaque, interpretable, and comprehensible systems—and introduces a fourth, truly explainable systems, based on a corpus analysis across multiple research fields.

Contribution

It offers a new conceptual framework for understanding explainable AI and introduces a novel category of truly explainable systems driven by automated reasoning.

Findings

Different research fields emphasize different notions of explainability.

A corpus analysis reveals how explainable AI is positioned across disciplines.

Introduction of a fourth notion: truly explainable systems with automated reasoning.

Abstract

We characterize three notions of explainable AI that cut across research fields: opaque systems that offer no insight into its algo- rithmic mechanisms; interpretable systems where users can mathemat- ically analyze its algorithmic mechanisms; and comprehensible systems that emit symbols enabling user-driven explanations of how a conclusion is reached. The paper is motivated by a corpus analysis of NIPS, ACL, COGSCI, and ICCV/ECCV paper titles showing differences in how work on explainable AI is positioned in various fields. We close by introducing a fourth notion: truly explainable systems, where automated reasoning is central to output crafted explanations without requiring human post processing as final step of the generative process.