The Impact of AI Assistance on Radiology Reporting: A Pilot Study Using Simulated AI Draft Reports

TL;DR

This pilot study demonstrates that AI-generated draft reports can significantly reduce radiology reporting time without compromising diagnostic accuracy, offering a practical solution to workload challenges.

Contribution

The study provides empirical evidence that AI-assisted reporting workflows can improve efficiency while maintaining accuracy in radiology reports.

Findings

AI drafts reduced reporting time by 25%

No significant difference in clinically significant errors

AI assistance accelerates workflow without accuracy loss

Abstract

Radiologists face increasing workload pressures amid growing imaging volumes, creating risks of burnout and delayed reporting times. While artificial intelligence (AI) based automated radiology report generation shows promise for reporting workflow optimization, evidence of its real-world impact on clinical accuracy and efficiency remains limited. This study evaluated the effect of draft reports on radiology reporting workflows by conducting a three reader multi-case study comparing standard versus AI-assisted reporting workflows. In both workflows, radiologists reviewed the cases and modified either a standard template (standard workflow) or an AI-generated draft report (AI-assisted workflow) to create the final report. For controlled evaluation, we used GPT-4 to generate simulated AI drafts and deliberately introduced 1-3 errors in half the cases to mimic real AI system performance. The AI-assisted workflow significantly reduced average reporting time from 573 to 435 seconds (p=0.003), without a statistically significant difference in clinically significant errors between workflows. These findings suggest that AI-generated drafts can meaningfully accelerate radiology reporting while maintaining diagnostic accuracy, offering a practical solution to address mounting workload challenges in clinical practice.