Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation

TL;DR

This paper introduces a cascaded diffusion model framework for synthesizing realistic 2D and 3D microscopy images with detailed annotations, significantly aiding cell segmentation tasks in biomedical research.

Contribution

It presents a novel multi-level diffusion approach combined with 3D surface reconstruction and texture synthesis to generate high-quality annotated microscopy images, reducing the need for manual annotation.

Findings

Training with synthetic data improves segmentation accuracy by up to 9%.

Synthetic images closely resemble real microscopy data based on FID scores.

The method effectively generates both 2D and 3D cell images with detailed annotations.

Abstract

Automated cell segmentation in microscopy images is essential for biomedical research, yet conventional methods are labor-intensive and prone to error. While deep learning-based approaches have proven effective, they often require large annotated datasets, which are scarce due to the challenges of manual annotation. To overcome this, we propose a novel framework for synthesizing densely annotated 2D and 3D cell microscopy images using cascaded diffusion models. Our method synthesizes 2D and 3D cell masks from sparse 2D annotations using multi-level diffusion models and NeuS, a 3D surface reconstruction approach. Following that, a pretrained 2D Stable Diffusion model is finetuned to generate realistic cell textures and the final outputs are combined to form cell populations. We show that training a segmentation model with a combination of our synthetic data and real data improves cell segmentation performance by up to 9\% across multiple datasets. Additionally, the FID scores indicate that the synthetic data closely resembles real data. The code for our proposed approach will be available at https://github.com/ruveydayilmaz0/cascaded_diffusion.