HIVE: Evaluating the Human Interpretability of Visual Explanations

TL;DR

This paper introduces HIVE, a novel human evaluation framework for visual explanations in AI, enabling systematic assessment of interpretability methods through large-scale human studies.

Contribution

HIVE is the first systematic human evaluation framework for visual interpretability methods, allowing comparison and hypothesis testing in AI explanations.

Findings

Explanations increase human trust in AI predictions.

Humans struggle to distinguish correct from incorrect explanations.

HIVE is open-source for future research.

Abstract

As AI technology is increasingly applied to high-impact, high-risk domains, there have been a number of new methods aimed at making AI models more human interpretable. Despite the recent growth of interpretability work, there is a lack of systematic evaluation of proposed techniques. In this work, we introduce HIVE (Human Interpretability of Visual Explanations), a novel human evaluation framework that assesses the utility of explanations to human users in AI-assisted decision making scenarios, and enables falsifiable hypothesis testing, cross-method comparison, and human-centered evaluation of visual interpretability methods. To the best of our knowledge, this is the first work of its kind. Using HIVE, we conduct IRB-approved human studies with nearly 1000 participants and evaluate four methods that represent the diversity of computer vision interpretability works: GradCAM, BagNet, ProtoPNet, and ProtoTree. Our results suggest that explanations engender human trust, even for incorrect predictions, yet are not distinct enough for users to distinguish between correct and incorrect predictions. We open-source HIVE to enable future studies and encourage more human-centered approaches to interpretability research.