Trustworthy Medical Imaging with Large Language Models: A Study of Hallucinations Across Modalities

TL;DR

This paper investigates hallucinations in large language models applied to medical imaging, analyzing errors in image-to-text and text-to-image tasks to improve clinical reliability and safety.

Contribution

It systematically studies hallucination patterns in medical imaging LLMs, providing insights into their causes and implications for enhancing trustworthiness.

Findings

Common hallucination patterns identified across modalities

Errors include factual inconsistencies and anatomical inaccuracies

Implications for improving model reliability in clinical settings

Abstract

Large Language Models (LLMs) are increasingly applied to medical imaging tasks, including image interpretation and synthetic image generation. However, these models often produce hallucinations, which are confident but incorrect outputs that can mislead clinical decisions. This study examines hallucinations in two directions: image to text, where LLMs generate reports from X-ray, CT, or MRI scans, and text to image, where models create medical images from clinical prompts. We analyze errors such as factual inconsistencies and anatomical inaccuracies, evaluating outputs using expert informed criteria across imaging modalities. Our findings reveal common patterns of hallucination in both interpretive and generative tasks, with implications for clinical reliability. We also discuss factors contributing to these failures, including model architecture and training data. By systematically studying both image understanding and generation, this work provides insights into improving the safety and trustworthiness of LLM driven medical imaging systems.