Predictive AI Can Support Human Learning while Preserving Error Diversity

TL;DR

This study investigates how predictive AI influences medical novices' learning and error diversity, revealing that combined AI deployment during training and practice enhances diagnostic accuracy and error distribution.

Contribution

It provides empirical evidence on the effects of AI deployment timing on individual learning and error diversity in medical diagnosis training.

Findings

AI during training or practice improves diagnostic accuracy

Combined AI deployment yields greater accuracy gains

AI deployment influences error diversity and group decision accuracy

Abstract

We examined the effects of predictive AI deployment on the immediate performance and learning of medical novices. In two pre-registered field experiments, we varied whether AI input was provided during the training or practice of lung cancer diagnoses, or both. Our results show that different AI deployments have distinct implications for human professionals. AI input during training or practice independently improves individuals' diagnostic accuracy, whereas deployment across both phases yields gains that exceed either approach alone. Furthermore, AI input in both training and earlier practice can improve the accuracy of individuals' subsequent independent diagnoses. Beyond individual accuracy, AI deployment affects the diversity of errors across individuals, with consequences for the accuracy of group decisions (e.g. when getting a second or third opinion on a diagnosis).