Mask-conditioned latent diffusion for generating gastrointestinal polyp images

TL;DR

This paper introduces a diffusion probabilistic model conditioned on segmentation masks to generate high-quality synthetic gastrointestinal polyp images, aiding in overcoming data scarcity in medical imaging.

Contribution

It presents a novel conditional diffusion model for generating synthetic GI polyp images with masks, improving data augmentation for medical image segmentation.

Findings

Synthetic images improve segmentation performance when combined with real data.

DeepLabv3+ achieves a micro-IOU of 0.7751 with combined real and synthetic data.

Model architecture influences the effectiveness of synthetic data in training.

Abstract

In order to take advantage of AI solutions in endoscopy diagnostics, we must overcome the issue of limited annotations. These limitations are caused by the high privacy concerns in the medical field and the requirement of getting aid from experts for the time-consuming and costly medical data annotation process. In computer vision, image synthesis has made a significant contribution in recent years as a result of the progress of generative adversarial networks (GANs) and diffusion probabilistic models (DPM). Novel DPMs have outperformed GANs in text, image, and video generation tasks. Therefore, this study proposes a conditional DPM framework to generate synthetic GI polyp images conditioned on given generated segmentation masks. Our experimental results show that our system can generate an unlimited number of high-fidelity synthetic polyp images with the corresponding ground truth masks of polyps. To test the usefulness of the generated data, we trained binary image segmentation models to study the effect of using synthetic data. Results show that the best micro-imagewise IOU of 0.7751 was achieved from DeepLabv3+ when the training data consists of both real data and synthetic data. However, the results reflect that achieving good segmentation performance with synthetic data heavily depends on model architectures.