Evaluating Physician-AI Interaction for Cancer Management: Paving the Path towards Precision Oncology

TL;DR

This study examines how physicians interact with ML-based decision support tools in cancer treatment, revealing tendencies toward over-reliance and automation bias despite awareness of model limitations.

Contribution

It provides empirical evidence on clinician decision-making behaviors when using ML and RCT evidence in simulated oncology scenarios.

Findings

Physicians show increased confidence with concordant ML and RCT data.

Most physicians favor ML-supported treatments when data are discordant.

Clinicians struggle to critically appraise ML model validation procedures.

Abstract

As machine learning (ML)-based decision support tools proliferate in clinical practice, understanding how clinicians integrate personalized ML predictions alongside randomized controlled trial (RCT) evidence is critical. We designed a web-based clinical decision support system (CDSS) presenting survival and adverse event data from a simulated RCT and ML model across 12 synthetic multiple myeloma scenarios. In a within- subjects study with 32 physicians, we evaluated how clinicians synthesize competing evidence sources to make treatment decisions. When ML and RCT outputs were concordant, physicians reported greater confidence than with RCT data alone. When results were discordant, most physicians shifted toward the ML-supported treatment, often before reviewing any information about model training or validation, suggesting a tendency toward automation bias rather than algorithm avoidance. Despite reporting higher perceived reliability after viewing model quality disclosures, physicians were largely unable to describe the validation procedures they had reviewed. Taken together, these findings reveal that clinicians may over-rely on ML recommendations even when equipped with tools designed to support critical appraisal. We discuss implications for CDSS design, clinician training, and the institutional safeguards needed before ML-based systems are deployed in high-stakes oncology settings.