MIQ-SAM3D: From Single-Point Prompt to Multi-Instance Segmentation via Competitive Query Refinement

TL;DR

MIQ-SAM3D introduces a multi-instance 3D segmentation framework that transforms single-point prompts into multiple lesion queries, improving multi-lesion segmentation accuracy and robustness in medical imaging.

Contribution

It proposes a novel competitive query refinement strategy and a hybrid CNN-Transformer encoder for enhanced multi-lesion segmentation from minimal prompts.

Findings

Achieved comparable segmentation performance on LiTS17 and KiTS21 datasets.

Demonstrated robustness to different prompts in multi-lesion segmentation.

Enabled efficient annotation of clinically relevant multi-lesion cases.

Abstract

Accurate segmentation of medical images is fundamental to tumor diagnosis and treatment planning. SAM-based interactive segmentation has gained attention for its strong generalization, but most methods follow a single-point-to-single-object paradigm, which limits multi-lesion segmentation. Moreover, ViT backbones capture global context but often miss high-fidelity local details. We propose MIQ-SAM3D, a multi-instance 3D segmentation framework with a competitive query optimization strategy that shifts from single-point-to-single-mask to single-point-to-multi-instance. A prompt-conditioned instance-query generator transforms a single point prompt into multiple specialized queries, enabling retrieval of all semantically similar lesions across the 3D volume from a single exemplar. A hybrid CNN-Transformer encoder injects CNN-derived boundary saliency into ViT self-attention via spatial gating. A competitively optimized query decoder then enables end-to-end, parallel, multi-instance prediction through inter-query competition. On LiTS17 and KiTS21 dataset, MIQ-SAM3D achieved comparable levels and exhibits strong robustness to prompts, providing a practical solution for efficient annotation of clinically relevant multi-lesion cases.