The Challenge of Crafting Intelligible Intelligence

TL;DR

This paper discusses the importance of making AI systems understandable, reviews recent approaches for interpretability, and emphasizes the need for developing methods to explain complex AI decisions in critical applications.

Contribution

It provides a comprehensive survey of recent work on AI interpretability and highlights future research directions for making AI more intelligible.

Findings

Interpretability is crucial for trust in AI systems.

Recent methods include local approximation and interactive explanations.

Future research should focus on developing more effective explanation techniques.

Abstract

Since Artificial Intelligence (AI) software uses techniques like deep lookahead search and stochastic optimization of huge neural networks to fit mammoth datasets, it often results in complex behavior that is difficult for people to understand. Yet organizations are deploying AI algorithms in many mission-critical settings. To trust their behavior, we must make AI intelligible, either by using inherently interpretable models or by developing new methods for explaining and controlling otherwise overwhelmingly complex decisions using local approximation, vocabulary alignment, and interactive explanation. This paper argues that intelligibility is essential, surveys recent work on building such systems, and highlights key directions for research.