Interactive Medical Image Segmentation: A Benchmark Dataset and Baseline

TL;DR

This paper introduces IMed-361M, a large-scale, diverse benchmark dataset for interactive medical image segmentation, along with a baseline model demonstrating improved accuracy and scalability across multiple modalities and annotation types.

Contribution

The paper presents the IMed-361M dataset with over 361 million masks across 14 modalities, and develops a baseline IMIS model leveraging foundational vision models for enhanced segmentation performance.

Findings

IMed-361M contains 361 million masks across 14 modalities.

The baseline model achieves superior accuracy and scalability.

The dataset and model are publicly available for research.

Abstract

Interactive Medical Image Segmentation (IMIS) has long been constrained by the limited availability of large-scale, diverse, and densely annotated datasets, which hinders model generalization and consistent evaluation across different models. In this paper, we introduce the IMed-361M benchmark dataset, a significant advancement in general IMIS research. First, we collect and standardize over 6.4 million medical images and their corresponding ground truth masks from multiple data sources. Then, leveraging the strong object recognition capabilities of a vision foundational model, we automatically generated dense interactive masks for each image and ensured their quality through rigorous quality control and granularity management. Unlike previous datasets, which are limited by specific modalities or sparse annotations, IMed-361M spans 14 modalities and 204 segmentation targets, totaling 361 million masks-an average of 56 masks per image. Finally, we developed an IMIS baseline network on this dataset that supports high-quality mask generation through interactive inputs, including clicks, bounding boxes, text prompts, and their combinations. We evaluate its performance on medical image segmentation tasks from multiple perspectives, demonstrating superior accuracy and scalability compared to existing interactive segmentation models. To facilitate research on foundational models in medical computer vision, we release the IMed-361M and model at https://github.com/uni-medical/IMIS-Bench.