Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT

TL;DR

This paper reviews the integration of large language models, especially ChatGPT, into medical imaging tasks, highlighting current applications, benefits, and future research directions to enhance clinical workflows and diagnostic accuracy.

Contribution

It provides a comprehensive overview of how language models are applied in medical imaging, emphasizing recent developments and potential applications like report generation and interpretability.

Findings

Language models improve medical image report accuracy

ChatGPT enables new diagnostic support tools

Potential to reduce clinical diagnostic errors

Abstract

In this paper, we aimed to provide a review and tutorial for researchers in the field of medical imaging using language models to improve their tasks at hand. We began by providing an overview of the history and concepts of language models, with a special focus on large language models. We then reviewed the current literature on how language models are being used to improve medical imaging, emphasizing different applications such as image captioning, report generation, report classification, finding extraction, visual question answering, interpretable diagnosis, and more for various modalities and organs. The ChatGPT was specially highlighted for researchers to explore more potential applications. We covered the potential benefits of accurate and efficient language models for medical imaging analysis, including improving clinical workflow efficiency, reducing diagnostic errors, and assisting healthcare professionals in providing timely and accurate diagnoses. Overall, our goal was to bridge the gap between language models and medical imaging and inspire new ideas and innovations in this exciting area of research. We hope that this review paper will serve as a useful resource for researchers in this field and encourage further exploration of the possibilities of language models in medical imaging.