Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domains

TL;DR

Pancakes is a novel framework that automatically generates consistent multi-protocol segmentations for biomedical images across different domains, outperforming existing models in producing semantically coherent results.

Contribution

Introduces a new problem formulation and a framework for multi-protocol segmentation across unseen biomedical image domains, ensuring semantic consistency.

Findings

Outperforms existing foundation models in multi-protocol segmentation

Produces semantically coherent segmentations across images

Effective on seven diverse biomedical datasets

Abstract

A single biomedical image can be meaningfully segmented in multiple ways, depending on the desired application. For instance, a brain MRI can be segmented according to tissue types, vascular territories, broad anatomical regions, fine-grained anatomy, or pathology, etc. Existing automatic segmentation models typically either (1) support only a single protocol, the one they were trained on, or (2) require labor-intensive manual prompting to specify the desired segmentation. We introduce Pancakes, a framework that, given a new image from a previously unseen domain, automatically generates multi-label segmentation maps for multiple plausible protocols, while maintaining semantic consistency across related images. Pancakes introduces a new problem formulation that is not currently attainable by existing foundation models. In a series of experiments on seven held-out datasets, we demonstrate that our model can significantly outperform existing foundation models in producing several plausible whole-image segmentations, that are semantically coherent across images.