Atomic Fact-Checking Increases Clinician Trust in Large Language Model Recommendations for Oncology Decision Support: A Randomized Controlled Trial

TL;DR

This study demonstrates that atomic fact-checking significantly enhances clinician trust in AI-driven oncology recommendations by decomposing suggestions into verifiable claims linked to source guidelines.

Contribution

It introduces atomic fact-checking as a novel approach that substantially improves trust over traditional explainability methods in clinical AI applications.

Findings

Atomic fact-checking increased clinician trust from 26.9% to 66.5%.

Large effect size (Cohen's d = 0.94) observed for atomic fact-checking.

Traditional transparency methods showed smaller, dose-response improvements.

Abstract

Question: Does atomic fact-checking, which decomposes AI treatment recommendations into individually verifiable claims linked to source guideline documents, increase clinician trust compared to traditional explainability approaches? Findings: In this randomized trial of 356 clinicians generating 7,476 trust ratings, atomic fact-checking produced a large effect on trust (Cohen's d = 0.94), increasing the proportion of clinicians expressing trust from 26.9% to 66.5%. Traditional transparency mechanisms showed a dose-response gradient of improvement over baseline (d = 0.25 to 0.50). Meaning: Decomposing AI recommendations into individually verifiable claims linked to source guidelines produces substantially higher clinician trust than traditional explainability approaches in high-stakes clinical decisions.