Scope and Sense of Explainability for AI-Systems

TL;DR

This paper critically examines the scope and challenges of explainability in AI systems, especially complex ones, arguing that some AI decisions may inherently defy classical explanation but are nonetheless valuable.

Contribution

It discusses the feasibility limits of explainability in complex AI systems and argues against discarding effective solutions that are not fully understandable.

Findings

Explainability is limited in highly complex AI systems.

Some AI solutions are unintelligible but still valuable.

Discarding non-explainable AI solutions wastes potential.

Abstract

Certain aspects of the explainability of AI systems will be critically discussed. This especially with focus on the feasibility of the task of making every AI system explainable. Emphasis will be given to difficulties related to the explainability of highly complex and efficient AI systems which deliver decisions whose explanation defies classical logical schemes of cause and effect. AI systems have provably delivered unintelligible solutions which in retrospect were characterized as ingenious (for example move 37 of the game 2 of AlphaGo). It will be elaborated on arguments supporting the notion that if AI-solutions were to be discarded in advance because of their not being thoroughly comprehensible, a great deal of the potentiality of intelligent systems would be wasted.