UniverSeg: Universal Medical Image Segmentation

TL;DR

UniverSeg is a universal medical image segmentation model that generalizes to unseen tasks without additional training by using a novel Cross-Block mechanism, trained on a large diverse dataset of 53 medical segmentation datasets.

Contribution

The paper introduces UniverSeg, a new method that enables zero-shot segmentation of unseen medical images using a novel Cross-Block mechanism and a large standardized dataset for training.

Findings

Outperforms related methods on unseen tasks

Generalizes across diverse anatomies and modalities

Uses a large, standardized dataset for training

Abstract

While deep learning models have become the predominant method for medical image segmentation, they are typically not capable of generalizing to unseen segmentation tasks involving new anatomies, image modalities, or labels. Given a new segmentation task, researchers generally have to train or fine-tune models, which is time-consuming and poses a substantial barrier for clinical researchers, who often lack the resources and expertise to train neural networks. We present UniverSeg, a method for solving unseen medical segmentation tasks without additional training. Given a query image and example set of image-label pairs that define a new segmentation task, UniverSeg employs a new Cross-Block mechanism to produce accurate segmentation maps without the need for additional training. To achieve generalization to new tasks, we have gathered and standardized a collection of 53 open-access medical segmentation datasets with over 22,000 scans, which we refer to as MegaMedical. We used this collection to train UniverSeg on a diverse set of anatomies and imaging modalities. We demonstrate that UniverSeg substantially outperforms several related methods on unseen tasks, and thoroughly analyze and draw insights about important aspects of the proposed system. The UniverSeg source code and model weights are freely available at https://universeg.csail.mit.edu