NULLBUS: Multimodal Mixed-Supervision for Breast Ultrasound Segmentation via Nullable Global-Local Prompts

TL;DR

NullBUS is a novel multimodal framework for breast ultrasound segmentation that effectively utilizes both prompt-based and prompt-free data, achieving state-of-the-art results despite missing metadata.

Contribution

It introduces nullable prompts with learnable null embeddings and presence masks, enabling robust learning from datasets with and without prompts in a single model.

Findings

Achieves a mean IoU of 0.8568 and Dice of 0.9103 on public datasets.

Demonstrates robustness in mixed prompt availability scenarios.

Sets new state-of-the-art performance in BUS segmentation.

Abstract

Breast ultrasound (BUS) segmentation provides lesion boundaries essential for computer-aided diagnosis and treatment planning. While promptable methods can improve segmentation performance and tumor delineation when text or spatial prompts are available, many public BUS datasets lack reliable metadata or reports, constraining training to small multimodal subsets and reducing robustness. We propose NullBUS, a multimodal mixed-supervision framework that learns from images with and without prompts in a single model. To handle missing text, we introduce nullable prompts, implemented as learnable null embeddings with presence masks, enabling fallback to image-only evidence when metadata are absent and the use of text when present. Evaluated on a unified pool of three public BUS datasets, NullBUS achieves a mean IoU of 0.8568 and a mean Dice of 0.9103, demonstrating state-of-the-art performance under mixed prompt availability.