Explainable AI And Visual Reasoning: Insights From Radiology

TL;DR

This paper investigates why current explainable AI in radiology fails to build trust, proposing that explanations mimicking human reasoning and evidence presentation could improve transparency and adoption.

Contribution

It introduces a human-centered explanation approach based on reasoning and evidence, demonstrated through a radiology case study, to enhance trust in AI predictions.

Findings

Practitioners rely on evidence-based explanations for trust.

Current AI explanations lack intuitive evidentiary grounding.

Human-like reasoning explanations may improve AI adoption.

Abstract

Why do explainable AI (XAI) explanations in radiology, despite their promise of transparency, still fail to gain human trust? Current XAI approaches provide justification for predictions, however, these do not meet practitioners' needs. These XAI explanations lack intuitive coverage of the evidentiary basis for a given classification, posing a significant barrier to adoption. We posit that XAI explanations that mirror human processes of reasoning and justification with evidence may be more useful and trustworthy than traditional visual explanations like heat maps. Using a radiology case study, we demonstrate how radiology practitioners get other practitioners to see a diagnostic conclusion's validity. Machine-learned classifications lack this evidentiary grounding and consequently fail to elicit trust and adoption by potential users. Insights from this study may generalize to guiding principles for human-centered explanation design based on human reasoning and justification of evidence.