Perceptual Evaluation of GANs and Diffusion Models for Generating X-rays

TL;DR

This study compares the perceptual quality of GANs and Diffusion Models in generating realistic chest X-ray images with abnormalities, highlighting their strengths and limitations for medical AI applications.

Contribution

It provides a comprehensive evaluation of GANs and DMs for medical image synthesis, including a reader study with radiologists and insights into perceptual differences.

Findings

Diffusion Models produce more realistic images overall.

GANs outperform in detecting certain abnormalities like absence of ECS.

Radiologists identify visual cues to distinguish synthetic from real images.

Abstract

Generative image models have achieved remarkable progress in both natural and medical imaging. In the medical context, these techniques offer a potential solution to data scarcity-especially for low-prevalence anomalies that impair the performance of AI-driven diagnostic and segmentation tools. However, questions remain regarding the fidelity and clinical utility of synthetic images, since poor generation quality can undermine model generalizability and trust. In this study, we evaluate the effectiveness of state-of-the-art generative models-Generative Adversarial Networks (GANs) and Diffusion Models (DMs)-for synthesizing chest X-rays conditioned on four abnormalities: Atelectasis (AT), Lung Opacity (LO), Pleural Effusion (PE), and Enlarged Cardiac Silhouette (ECS). Using a benchmark composed of real images from the MIMIC-CXR dataset and synthetic images from both GANs and DMs, we conducted a reader study with three radiologists of varied experience. Participants were asked to distinguish real from synthetic images and assess the consistency between visual features and the target abnormality. Our results show that while DMs generate more visually realistic images overall, GANs can report better accuracy for specific conditions, such as absence of ECS. We further identify visual cues radiologists use to detect synthetic images, offering insights into the perceptual gaps in current models. These findings underscore the complementary strengths of GANs and DMs and point to the need for further refinement to ensure generative models can reliably augment training datasets for AI diagnostic systems.