Towards 3D Semantic Image Synthesis for Medical Imaging

TL;DR

This paper introduces Med-LSDM, a 3D semantic image synthesis model for medical imaging that generates high-quality volumetric data using a diffusion process in latent space, aiding privacy and data augmentation.

Contribution

The study presents Med-LSDM, a novel 3D semantic image synthesis method leveraging latent diffusion and a guiding mechanism, specifically designed for medical volumetric data.

Findings

Achieves a 3D-FID score of 0.0054 on Duke Breast dataset.

Produces synthetic images with Dice scores close to real data.

Demonstrates effective data augmentation for medical imaging.

Abstract

In the medical domain, acquiring large datasets is challenging due to both accessibility issues and stringent privacy regulations. Consequently, data availability and privacy protection are major obstacles to applying machine learning in medical imaging. To address this, our study proposes the Med-LSDM (Latent Semantic Diffusion Model), which operates directly in the 3D domain and leverages de-identified semantic maps to generate synthetic data as a method of privacy preservation and data augmentation. Unlike many existing methods that focus on generating 2D slices, Med-LSDM is designed specifically for 3D semantic image synthesis, making it well-suited for applications requiring full volumetric data. Med-LSDM incorporates a guiding mechanism that controls the 3D image generation process by applying a diffusion model within the latent space of a pre-trained VQ-GAN. By operating in the compressed latent space, the model significantly reduces computational complexity while still preserving critical 3D spatial details. Our approach demonstrates strong performance in 3D semantic medical image synthesis, achieving a 3D-FID score of 0.0054 on the conditional Duke Breast dataset and similar Dice scores (0.70964) to those of real images (0.71496). These results demonstrate that the synthetic data from our model have a small domain gap with real data and are useful for data augmentation.