Integrating AI into Radiology workflow: Levels of research, production, and feedback maturity

TL;DR

This paper presents a structured roadmap for integrating AI into radiology workflows through three maturity levels, emphasizing research, production, and feedback to enhance AI performance with radiologist input.

Contribution

It introduces a comprehensive framework with three maturity levels for AI integration in radiology, including a case study demonstrating continuous improvement through radiologist feedback.

Findings

AI significantly reduces false positives in brain metastases detection.

Radiologist feedback increases annotated datasets for AI retraining.

Deployment improves AI accuracy over time.

Abstract

This report represents a roadmap for integrating Artificial Intelligence (AI)-based image analysis algorithms into existing Radiology workflows such that: (1) radiologists can significantly benefit from enhanced automation in various imaging tasks due to AI; and (2) radiologists' feedback is utilized to further improve the AI application. This is achieved by establishing three maturity levels where: (1) research enables the visualization of AI-based results/annotations by radiologists without generating new patient records; (2) production allows the AI-based system to generate results stored in an institution's Picture Archiving and Communication System; and (3) feedback equips radiologists with tools for editing the AI inference results for periodic retraining of the deployed AI systems, thereby allowing the continuous organic improvement of AI-based radiology-workflow solutions. A case study (i.e., detection of brain metastases with T1-weighted contrast-enhanced 3D MRI) illustrates the deployment details of a particular AI-based application according to the aforementioned maturity levels. It is shown that the given AI application significantly improves with the feedback coming from radiologists; the number of incorrectly detected brain metastases (false positives) reduces from 14.2 to 9.12 per patient with the number of subsequently annotated datasets increasing from 93 to 217 as a result of radiologist adjudication.