UltraSam: A Foundation Model for Ultrasound using Large Open-Access Segmentation Datasets

TL;DR

UltraSam is a new foundation model trained on the largest open-access ultrasound dataset, significantly improving segmentation performance and serving as a versatile starting point for various ultrasound analysis tasks.

Contribution

The paper introduces US-43d, the largest open-access ultrasound dataset, and UltraSam, a SAM-based foundation model tailored for ultrasound image segmentation and analysis.

Findings

UltraSam outperforms existing SAM-style models in prompt-based segmentation.

UltraSam-initialized models surpass other pretraining methods in downstream tasks.

The dataset US-43d enables large-scale ultrasound model training.

Abstract

Purpose: Automated ultrasound image analysis is challenging due to anatomical complexity and limited annotated data. To tackle this, we take a data-centric approach, assembling the largest public ultrasound segmentation dataset and training a versatile visual foundation model tailored for ultrasound. Methods: We compile US-43d, a large-scale collection of 43 open-access ultrasound datasets with over 280,000 images and segmentation masks for more than 50 anatomical structures. We then introduce UltraSam, an adaptation of the Segment Anything Model (SAM) that is trained on US-43d and supports both point- and box-prompts. Finally, we introduce a new use case for SAM-style models by using UltraSam as a model initialization that can be fine-tuned for various downstream analysis tasks, demonstrating UltraSam's foundational capabilities. Results: UltraSam achieves vastly improved performance over existing SAM-style models for prompt-based segmentation on three diverse public datasets. Moreover, an UltraSam-initialized Vision Transformer surpasses ImageNet-, SAM-, and MedSAM-initialized models in various downstream segmentation and classification tasks, highlighting UltraSam's effectiveness as a foundation model. Conclusion: We compile US-43d, a large-scale unified ultrasound dataset, and introduce UltraSam, a powerful multi-purpose SAM-style model for ultrasound images. We release our code and pretrained models at https://github.com/CAMMA-public/UltraSam and invite the community to further this effort by contributing high-quality datasets.