Promptable Counterfactual Diffusion Model for Unified Brain Tumor Segmentation and Generation with MRIs

TL;DR

This paper introduces a promptable counterfactual diffusion model that unifies brain tumor segmentation and generation in MRI, enabling guided manipulation of tumor regions and improving accuracy over traditional methods.

Contribution

The novel model incorporates mask-level prompting for bidirectional inference, allowing simultaneous tumor segmentation and realistic tumor generation in MRI images.

Findings

Achieves a mean IoU of 0.653 for tumor segmentation

Attains a mean Dice score of 0.785, outperforming traditional approaches

Demonstrates versatility in tumor synthesis and position transfer

Abstract

Brain tumor analysis in Magnetic Resonance Imaging (MRI) is crucial for accurate diagnosis and treatment planning. However, the task remains challenging due to the complexity and variability of tumor appearances, as well as the scarcity of labeled data. Traditional approaches often address tumor segmentation and image generation separately, limiting their effectiveness in capturing the intricate relationships between healthy and pathological tissue structures. We introduce a novel promptable counterfactual diffusion model as a unified solution for brain tumor segmentation and generation in MRI. The key innovation lies in our mask-level prompting mechanism at the sampling stage, which enables guided generation and manipulation of specific healthy or unhealthy regions in MRI images. Specifically, the model's architecture allows for bidirectional inference, which can segment tumors in existing images and generate realistic tumor structures in healthy brain scans. Furthermore, we present a two-step approach for tumor generation and position transfer, showcasing the model's versatility in synthesizing realistic tumor structures. Experiments on the BRATS2021 dataset demonstrate that our method outperforms traditional counterfactual diffusion approaches, achieving a mean IoU of 0.653 and mean Dice score of 0.785 for tumor segmentation, outperforming the 0.344 and 0.475 of conventional counterfactual diffusion model. Our work contributes to improving brain tumor detection and segmentation accuracy, with potential implications for data augmentation and clinical decision support in neuro-oncology. The code is available at https://github.com/arcadelab/counterfactual_diffusion.