ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models

TL;DR

This paper introduces ArSDM, a novel diffusion model that generates high-quality colonoscopy images conditioned on segmentation masks, improving data augmentation for better polyp detection and segmentation.

Contribution

The paper presents ArSDM, an adaptive diffusion model that refines generated images using segmentation masks and size-based loss adjustment, enhancing medical image synthesis.

Findings

Generated data significantly improves baseline segmentation performance.

ArSDM produces high-quality, realistic colonoscopy images.

The method effectively reduces mask prediction errors during training.

Abstract

Colonoscopy analysis, particularly automatic polyp segmentation and detection, is essential for assisting clinical diagnosis and treatment. However, as medical image annotation is labour- and resource-intensive, the scarcity of annotated data limits the effectiveness and generalization of existing methods. Although recent research has focused on data generation and augmentation to address this issue, the quality of the generated data remains a challenge, which limits the contribution to the performance of subsequent tasks. Inspired by the superiority of diffusion models in fitting data distributions and generating high-quality data, in this paper, we propose an Adaptive Refinement Semantic Diffusion Model (ArSDM) to generate colonoscopy images that benefit the downstream tasks. Specifically, ArSDM utilizes the ground-truth segmentation mask as a prior condition during training and adjusts the diffusion loss for each input according to the polyp/background size ratio. Furthermore, ArSDM incorporates a pre-trained segmentation model to refine the training process by reducing the difference between the ground-truth mask and the prediction mask. Extensive experiments on segmentation and detection tasks demonstrate the generated data by ArSDM could significantly boost the performance of baseline methods.