Beyond Satisfaction: From Placebic to Actionable Explanations For Enhanced Understandability
Joe Shymanski, Jacob Brue, Sandip Sen

TL;DR
This paper critiques the reliance on subjective satisfaction in evaluating explainable AI, demonstrating that actionable explanations improve objective understanding even when user ratings do not differ from vacuous explanations.
Contribution
It introduces an experimental framework that distinguishes meaningful explanations from placebic ones and advocates for combining objective metrics with subjective surveys in XAI evaluation.
Findings
Participants with actionable explanations showed better mental models.
Subjective satisfaction ratings did not differ between explanation types.
Objective measures reveal the true effectiveness of explanations.
Abstract
Explainable AI (XAI) presents useful tools to facilitate transparency and trustworthiness in machine learning systems. However, current evaluations of system explainability often rely heavily on subjective user surveys, which may not adequately capture the effectiveness of explanations. This paper critiques the overreliance on user satisfaction metrics and explores whether these can differentiate between meaningful (actionable) and vacuous (placebic) explanations. In experiments involving optimal Social Security filing age selection tasks, participants used one of three protocols: no explanations, placebic explanations, and actionable explanations. Participants who received actionable explanations significantly outperformed the other groups in objective measures of their mental model, but users rated placebic and actionable explanations as equally satisfying. This suggests that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsExplainable Artificial Intelligence (XAI) · Artificial Intelligence in Healthcare and Education · Ethics and Social Impacts of AI
