Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making

TL;DR

This paper explores distance-based uncertainty scores for AI task delegation, demonstrating their superiority over probability-based scores in improving human decision-making accuracy and reliance in a healthcare setting.

Contribution

It introduces the use of distance-based uncertainty scores and visualization techniques to enhance human-AI collaboration and decision accuracy.

Findings

Distance-based uncertainty scores outperform probability-based scores in identifying uncertain cases.

Participants showed improved decision accuracy and confidence after reviewing distance-based scores.

The approach significantly increased correct decisions and reduced incorrect changes ($p<0.01$).

Abstract

Despite the growing promise of artificial intelligence (AI) in supporting decision-making across domains, fostering appropriate human reliance on AI remains a critical challenge. In this paper, we investigate the utility of exploring distance-based uncertainty scores for task delegation to AI and describe how these scores can be visualized through embedding representations for human-AI decision-making. After developing an AI-based system for physical stroke rehabilitation assessment, we conducted a study with 19 health professionals and 10 students in medicine/health to understand the effect of exploring distance-based uncertainty scores on users' reliance on AI. Our findings showed that distance-based uncertainty scores outperformed traditional probability-based uncertainty scores in identifying uncertain cases. In addition, after exploring confidence scores for task delegation and reviewing embedding-based visualizations of distance-based uncertainty scores, participants achieved an 8.20% higher rate of correct decisions, a 7.15% higher rate of changing their decisions to correct ones, and a 7.14% lower rate of incorrect changes after reviewing AI outputs than those reviewing probability-based uncertainty scores ($p<0.01$). Our findings highlight the potential of distance-based uncertainty scores to enhance decision accuracy and appropriate reliance on AI while discussing ongoing challenges for human-AI collaborative decision-making.