Chest-Diffusion: A Light-Weight Text-to-Image Model for Report-to-CXR Generation

TL;DR

Chest-Diffusion is a lightweight, transformer-based diffusion model that generates realistic chest X-ray images from reports, significantly reducing computational costs while improving image authenticity.

Contribution

It introduces a domain-specific text encoder and a lightweight transformer denoising architecture for report-to-CXR generation, enhancing efficiency and authenticity.

Findings

Achieves lowest FID score of 24.456

Uses nearly one-third the computational complexity of Stable Diffusion

Demonstrates effective report-to-CXR image generation

Abstract

Text-to-image generation has important implications for generation of diverse and controllable images. Several attempts have been made to adapt Stable Diffusion (SD) to the medical domain. However, the large distribution difference between medical reports and natural texts, as well as high computational complexity in common stable diffusion limit the authenticity and feasibility of the generated medical images. To solve above problems, we propose a novel light-weight transformer-based diffusion model learning framework, Chest-Diffusion, for report-to-CXR generation. Chest-Diffusion employs a domain-specific text encoder to obtain accurate and expressive text features to guide image generation, improving the authenticity of the generated images. Meanwhile, we introduce a light-weight transformer architecture as the denoising model, reducing the computational complexity of the diffusion model. Experiments demonstrate that our Chest-Diffusion achieves the lowest FID score 24.456, under the computation budget of 118.918 GFLOPs, which is nearly one-third of the computational complexity of SD.