Importance of User Control in Data-Centric Steering for Healthcare Experts

TL;DR

This study investigates how different levels of user control in data-centric AI model tuning impact healthcare experts' trust, understanding, and model performance, highlighting the benefits of manual control and proposing hybrid approaches.

Contribution

It is the first to empirically compare manual and automated data-centric steering approaches in healthcare, demonstrating the advantages of manual control for trust and performance.

Findings

Manual steering improves model performance.

Manual control maintains trust and understandability.

Hybrid systems can enhance human-AI collaboration.

Abstract

As Artificial Intelligence (AI) becomes increasingly integrated into high-stakes domains like healthcare, effective collaboration between healthcare experts and AI systems is critical. Data-centric steering, which involves fine-tuning prediction models by improving training data quality, plays a key role in this process. However, little research has explored how varying levels of user control affect healthcare experts during data-centric steering. We address this gap by examining manual and automated steering approaches through a between-subjects, mixed-methods user study with 74 healthcare experts. Our findings show that manual steering, which grants direct control over training data, significantly improves model performance while maintaining trust and system understandability. Based on these findings, we propose design implications for a hybrid steering system that combines manual and automated approaches to increase user involvement during human-AI collaboration.