Foundation Model-guided Iteratively Prompting and Pseudo-Labeling for Partially Labeled Medical Image Segmentation

TL;DR

This paper introduces IPnP, a framework that iteratively refines pseudo-labels for partially labeled medical images by combining a trainable network and a foundation model, improving segmentation accuracy.

Contribution

The novel IPnP framework effectively addresses partial labeling in medical image segmentation by collaborative pseudo-labeling and prompting, approaching fully labeled performance.

Findings

IPnP outperforms prior methods on the AMOS dataset.

IPnP approaches fully labeled performance in experiments.

Effective in real-world clinical datasets.

Abstract

Automated medical image segmentation has achieved remarkable progress with fully labeled data. However, site-specific clinical priorities and the high cost of manual annotation often yield scans with only a subset of organs labeled, leading to the partially labeled problem that degrades performance. To address this issue, we propose IPnP, an Iteratively Prompting and Pseudo-labeling framework, for partially labeled medical image segmentation. IPnP iteratively generates and refines pseudo-labels for unlabeled organs through collaboration between a trainable segmentation network (specialist) and a frozen foundation model (generalist), progressively recovering full-organ supervision. On the public dataset AMOS with the simulated partial-label setting, IPnP consistently improves segmentation performance over prior methods and approaches the performance of the fully labeled reference. We further evaluate on a private, partially labeled dataset of 210 head-and-neck cancer patients and demonstrate our effectiveness in real-world clinical settings.