Machine Explanations and Human Understanding
Chacha Chen, Shi Feng, Amit Sharma, Chenhao Tan

TL;DR
This paper formalizes how explanations influence human understanding of machine learning models, emphasizing the role of human intuitions and providing a framework for improving explanations to better support human-AI decision making.
Contribution
It introduces a formal causal diagram framework to analyze the conditions under which explanations enhance human understanding, highlighting the importance of human intuitions.
Findings
Explanations can improve understanding of model decision boundary with assumptions about human intuitions.
Without assumptions, explanations cannot improve understanding of task decision boundary or model error.
Empirical studies confirm the critical role of human feature relevance intuitions in detecting model errors.
Abstract
Explanations are hypothesized to improve human understanding of machine learning models and achieve a variety of desirable outcomes, ranging from model debugging to enhancing human decision making. However, empirical studies have found mixed and even negative results. An open question, therefore, is under what conditions explanations can improve human understanding and in what way. Using adapted causal diagrams, we provide a formal characterization of the interplay between machine explanations and human understanding, and show how human intuitions play a central role in enabling human understanding. Specifically, we identify three core concepts of interest that cover all existing quantitative measures of understanding in the context of human-AI decision making: task decision boundary, model decision boundary, and model error. Our key result is that without assumptions about…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsExplainable Artificial Intelligence (XAI) · Bayesian Modeling and Causal Inference · Adversarial Robustness in Machine Learning
