Are Good Explainers Secretly Human-in-the-Loop Active Learners?

TL;DR

This paper explores how explainable AI techniques can be viewed as human-in-the-loop active learning, providing a formal framework to compare and evaluate their effectiveness through simulations instead of costly user studies.

Contribution

It introduces a mathematical approximation and formalization of explainable AI as active learning, enabling rigorous comparison and simulation-based utility assessment.

Findings

Initial promising results demonstrate potential benefits.

Framework allows comparison with standard active learning.

Simulation approach reduces need for expensive user studies.

Abstract

Explainable AI (XAI) techniques have become popular for multiple use-cases in the past few years. Here we consider its use in studying model predictions to gather additional training data. We argue that this is equivalent to Active Learning, where the query strategy involves a human-in-the-loop. We provide a mathematical approximation for the role of the human, and present a general formalization of the end-to-end workflow. This enables us to rigorously compare this use with standard Active Learning algorithms, while allowing for extensions to the workflow. An added benefit is that their utility can be assessed via simulation instead of conducting expensive user-studies. We also present some initial promising results.