Improving Health Professionals' Onboarding with AI and XAI for Trustworthy Human-AI Collaborative Decision Making

TL;DR

This study explores how to effectively introduce AI and explainable AI to health professionals with limited technical background, emphasizing practical benefits, benchmarks, and interaction trials to foster trust and collaboration.

Contribution

The paper provides empirical insights into onboarding strategies for AI and XAI tailored for health professionals, based on interviews and human-AI interaction guidelines.

Findings

Participants want benchmark info and practical AI benefits.

Interaction trials help contextualize AI performance.

Effective onboarding improves human-AI collaborative decision-making.

Abstract

With advanced AI/ML, there has been growing research on explainable AI (XAI) and studies on how humans interact with AI and XAI for effective human-AI collaborative decision-making. However, we still have a lack of understanding of how AI systems and XAI should be first presented to users without technical backgrounds. In this paper, we present the findings of semi-structured interviews with health professionals (n=12) and students (n=4) majoring in medicine and health to study how to improve onboarding with AI and XAI. For the interviews, we built upon human-AI interaction guidelines to create onboarding materials of an AI system for stroke rehabilitation assessment and AI explanations and introduce them to the participants. Our findings reveal that beyond presenting traditional performance metrics on AI, participants desired benchmark information, the practical benefits of AI, and interaction trials to better contextualize AI performance, and refine the objectives and performance of AI. Based on these findings, we highlight directions for improving onboarding with AI and XAI and human-AI collaborative decision-making.