Semantic Segmentation Refiner for Ultrasound Applications with Zero-Shot Foundation Models

TL;DR

This paper introduces a zero-shot, prompt-less segmentation refinement method for ultrasound images that leverages foundation models and coarse masks, significantly improving performance in low-data scenarios.

Contribution

The novel prompt point generation algorithm enables effective zero-shot segmentation refinement without additional prompts, addressing data scarcity and domain gap issues.

Findings

Significant performance gains in low-data regimes

Effective segmentation of pathological anomalies in ultrasound

Robustness across varying training set sizes

Abstract

Despite the remarkable success of deep learning in medical imaging analysis, medical image segmentation remains challenging due to the scarcity of high-quality labeled images for supervision. Further, the significant domain gap between natural and medical images in general and ultrasound images in particular hinders fine-tuning models trained on natural images to the task at hand. In this work, we address the performance degradation of segmentation models in low-data regimes and propose a prompt-less segmentation method harnessing the ability of segmentation foundation models to segment abstract shapes. We do that via our novel prompt point generation algorithm which uses coarse semantic segmentation masks as input and a zero-shot prompt-able foundation model as an optimization target. We demonstrate our method on a segmentation findings task (pathologic anomalies) in ultrasound images. Our method's advantages are brought to light in varying degrees of low-data regime experiments on a small-scale musculoskeletal ultrasound images dataset, yielding a larger performance gain as the training set size decreases.