XReal: Realistic Anatomy and Pathology-Aware X-ray Generation via Controllable Diffusion Model

TL;DR

XReal is a controllable diffusion model that generates realistic chest X-ray images with precise anatomy and pathology placement, improving over existing models in realism and clinical relevance without requiring model fine-tuning.

Contribution

It introduces a novel anatomy and pathology control mechanism for diffusion models, enabling precise spatial control in medical image generation without fine-tuning.

Findings

Outperforms state-of-the-art X-ray diffusion models in quantitative metrics.

Achieves higher radiologist ratings for realism.

Demonstrates significant improvements in anatomical and pathological accuracy.

Abstract

Large-scale generative models have demonstrated impressive capabilities in producing visually compelling images, with increasing applications in medical imaging. However, they continue to grapple with hallucination challenges and the generation of anatomically inaccurate outputs. These limitations are mainly due to the reliance on textual inputs and lack of spatial control over the generated images, hindering the potential usefulness of such models in real-life settings. In this work, we present XReal, a novel controllable diffusion model for generating realistic chest X-ray images through precise anatomy and pathology location control. Our lightweight method comprises an Anatomy Controller and a Pathology Controller to introduce spatial control over anatomy and pathology in a pre-trained Text-to-Image Diffusion Model, respectively, without fine-tuning the model. XReal outperforms state-of-the-art X-ray diffusion models in quantitative metrics and radiologists' ratings, showing significant gains in anatomy and pathology realism. Our model holds promise for advancing generative models in medical imaging, offering greater precision and adaptability while inviting further exploration in this evolving field. The code and pre-trained model weights are publicly available at https://github.com/BioMedIA-MBZUAI/XReal.