Beyond Pixel-Wise Supervision for Medical Image Segmentation: From Traditional Models to Foundation Models

TL;DR

This paper surveys recent advances in medical image segmentation that leverage weak annotations and foundation models like SAM to reduce annotation effort and improve segmentation performance.

Contribution

It provides a comprehensive review of annotation-efficient methods and discusses challenges and future directions in integrating foundation models into medical image segmentation.

Findings

Foundation models enable promptable segmentation with weak annotations.

Weakly supervised methods reduce reliance on pixel-wise annotations.

Challenges include data heterogeneity and model generalization.

Abstract

Medical image segmentation plays an important role in many image-guided clinical approaches. However, existing segmentation algorithms mostly rely on the availability of fully annotated images with pixel-wise annotations for training, which can be both labor-intensive and expertise-demanding, especially in the medical imaging domain where only experts can provide reliable and accurate annotations. To alleviate this challenge, there has been a growing focus on developing segmentation methods that can train deep models with weak annotations, such as image-level, bounding boxes, scribbles, and points. The emergence of vision foundation models, notably the Segment Anything Model (SAM), has introduced innovative capabilities for segmentation tasks using weak annotations for promptable segmentation enabled by large-scale pre-training. Adopting foundation models together with traditional learning methods has increasingly gained recent interest research community and shown potential for real-world applications. In this paper, we present a comprehensive survey of recent progress on annotation-efficient learning for medical image segmentation utilizing weak annotations before and in the era of foundation models. Furthermore, we analyze and discuss several challenges of existing approaches, which we believe will provide valuable guidance for shaping the trajectory of foundational models to further advance the field of medical image segmentation.