Harnessing Explanations to Bridge AI and Humans

TL;DR

This paper discusses the gap between current AI explanation methods and their actual effectiveness in improving human decision-making, proposing future research directions to bridge this gap.

Contribution

It highlights the disconnect between explanation quality and human decision improvement, suggesting new directions for research to enhance AI-human collaboration.

Findings

Explanations often do not improve human decision-making as expected

Current interpretability methods need better alignment with human cognitive processes

Future research should focus on closing the gap between explanation quality and decision support

Abstract

Machine learning models are increasingly integrated into societally critical applications such as recidivism prediction and medical diagnosis, thanks to their superior predictive power. In these applications, however, full automation is often not desired due to ethical and legal concerns. The research community has thus ventured into developing interpretable methods that explain machine predictions. While these explanations are meant to assist humans in understanding machine predictions and thereby allowing humans to make better decisions, this hypothesis is not supported in many recent studies. To improve human decision-making with AI assistance, we propose future directions for closing the gap between the efficacy of explanations and improvement in human performance.