A User-Centric Analysis of Explainability in AI-Based Medical Image Diagnosis

TL;DR

This study evaluates various explainability methods in AI medical diagnosis, highlighting user preferences, effectiveness, and risks of false diagnoses through a survey of physicians.

Contribution

It provides a comparative analysis of textual, visual, and multimodal XAI methods in medical imaging, emphasizing user trust and understanding.

Findings

Physicians highly value AI explanations for diagnoses.

Combined bounding box and report methods outperform others in key aspects.

Half of the physicians trusted false AI diagnoses over explainability methods.

Abstract

In recent years, AI systems in the medical domain have advanced significantly. However, despite outperforming humans, they are rarely used in practice since it is often not clear how they make their decisions. Optimal explanation and visualization of the decision process are often lacking. Therefore, we conducted a comparative user-centric analysis of the latest state-of-the-art textual, visual and multimodal explainable artificial intelligence (XAI) methods for medical image diagnosis. Our survey of 33 physicians showed that 88% agree that it is important that AI explains the diagnosis -- 64% even strongly agree. A combination of bounding box and report is rated better than the other tested XAI methods in the evaluated aspects understandability, completeness, speed, and applicability. We even tested the potential negative impact of false AI-based medical image diagnoses and found that 50% of the participants trusted false AI diagnoses over all tested XAI methods.