Foundation Models in Radiology: What, How, When, Why and Why Not

TL;DR

This paper reviews the emergence of foundation models in radiology, discussing their training, capabilities, evaluation, and potential to transform clinical practice while addressing challenges and safety considerations.

Contribution

It establishes standardized terminology and outlines pathways for developing radiology-specific foundation models, emphasizing benefits and challenges.

Findings

Foundation models can interpret and generate radiological data effectively.

Training data and evaluation strategies are crucial for model performance.

Addressing safety and ethical concerns is essential for clinical deployment.

Abstract

Recent advances in artificial intelligence have witnessed the emergence of large-scale deep learning models capable of interpreting and generating both textual and imaging data. Such models, typically referred to as foundation models, are trained on extensive corpora of unlabeled data and demonstrate high performance across various tasks. Foundation models have recently received extensive attention from academic, industry, and regulatory bodies. Given the potentially transformative impact that foundation models can have on the field of radiology, this review aims to establish a standardized terminology concerning foundation models, with a specific focus on the requirements of training data, model training paradigms, model capabilities, and evaluation strategies. We further outline potential pathways to facilitate the training of radiology-specific foundation models, with a critical emphasis on elucidating both the benefits and challenges associated with such models. Overall, we envision that this review can unify technical advances and clinical needs in the training of foundation models for radiology in a safe and responsible manner, for ultimately benefiting patients, providers, and radiologists.