Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence

TL;DR

This paper proposes a normative framework inspired by philosophy of science and cognitive science to evaluate explainability techniques in AI, aiming to address the opacity of machine learning systems and meet stakeholders' explanatory needs.

Contribution

It introduces a novel normative framework based on David Marr's explanation model to assess the effectiveness of current explainability methods in AI.

Findings

Framework clarifies different explanatory questions in AI systems

Enables evaluation of techniques like heatmapping and feature detection

Assesses the extent to which the Black Box Problem can be addressed

Abstract

Many of the computing systems programmed using Machine Learning are opaque: it is difficult to know why they do what they do or how they work. The Explainable Artificial Intelligence research program aims to develop analytic techniques with which to render opaque computing systems transparent, but lacks a normative framework with which to evaluate these techniques' explanatory success. The aim of the present discussion is to develop such a framework, while paying particular attention to different stakeholders' distinct explanatory requirements. Building on an analysis of 'opacity' from philosophy of science, this framework is modeled after David Marr's influential account of explanation in cognitive science. Thus, the framework distinguishes between the different questions that might be asked about an opaque computing system, and specifies the general way in which these questions should be answered. By applying this normative framework to current techniques such as input heatmapping, feature-detector identification, and diagnostic classification, it will be possible to determine whether and to what extent the Black Box Problem can be solved.