Diffusion-Based Data Augmentation for Medical Image Segmentation

TL;DR

This paper introduces DiffAug, a diffusion-based data augmentation framework that synthesizes realistic medical abnormalities guided by text prompts, improving segmentation performance especially on rare cases.

Contribution

The paper presents a novel diffusion-based augmentation method combining text-guided synthesis and validation for medical image segmentation.

Findings

Achieves 8-10% Dice score improvements over baselines.

Reduces false negatives by up to 28% in challenging cases.

Validates synthetic samples efficiently through latent space segmentation.

Abstract

Medical image segmentation models struggle with rare abnormalities due to scarce annotated pathological data. We propose DiffAug a novel framework that combines textguided diffusion-based generation with automatic segmentation validation to address this challenge. Our proposed approach uses latent diffusion models conditioned on medical text descriptions and spatial masks to synthesize abnormalities via inpainting on normal images. Generated samples undergo dynamic quality validation through a latentspace segmentation network that ensures accurate localization while enabling single-step inference. The text prompts, derived from medical literature, guide the generation of diverse abnormality types without requiring manual annotation. Our validation mechanism filters synthetic samples based on spatial accuracy, maintaining quality while operating efficiently through direct latent estimation. Evaluated on three medical imaging benchmarks (CVC-ClinicDB, Kvasir-SEG, REFUGE2), our framework achieves state-of-the-art performance with 8-10% Dice improvements over baselines and reduces false negative rates by up to 28% for challenging cases like small polyps and flat lesions critical for early detection in screening applications.