Lightweight Method for Interactive 3D Medical Image Segmentation with Multi-Round Result Fusion

TL;DR

LIM-Net is a lightweight CNN-based model for interactive 3D medical image segmentation that achieves strong generalization, efficiency, and competitive accuracy through multi-round result fusion, offering a practical alternative to SAM-based approaches.

Contribution

The paper introduces LIM-Net, a compact CNN model with multi-round result fusion, demonstrating superior generalization and efficiency in 3D medical image segmentation compared to existing SAM-based methods.

Findings

LIM-Net outperforms SAM-based models in generalization to unseen data.

LIM-Net requires fewer user interactions for accurate segmentation.

LIM-Net has low GPU memory usage, suitable for resource-limited environments.

Abstract

In medical imaging, precise annotation of lesions or organs is often required. However, 3D volumetric images typically consist of hundreds or thousands of slices, making the annotation process extremely time-consuming and laborious. Recently, the Segment Anything Model (SAM) has drawn widespread attention due to its remarkable zero-shot generalization capabilities in interactive segmentation. While researchers have explored adapting SAM for medical applications, such as using SAM adapters or constructing 3D SAM models, a key question remains: Can traditional CNN networks achieve the same strong zero-shot generalization in this task? In this paper, we propose the Lightweight Interactive Network for 3D Medical Image Segmentation (LIM-Net), a novel approach demonstrating the potential of compact CNN-based models. Built upon a 2D CNN backbone, LIM-Net initiates segmentation by generating a 2D prompt mask from user hints. This mask is then propagated through the 3D sequence via the Memory Module. To refine and stabilize results during interaction, the Multi-Round Result Fusion (MRF) Module selects and merges optimal masks from multiple rounds. Our extensive experiments across multiple datasets and modalities demonstrate LIM-Net's competitive performance. It exhibits stronger generalization to unseen data compared to SAM-based models, with competitive accuracy while requiring fewer interactions. Notably, LIM-Net's lightweight design offers significant advantages in deployment and inference efficiency, with low GPU memory consumption suitable for resource-constrained environments. These promising results demonstrate LIM-Net can serve as a strong baseline, complementing and contrasting with popular SAM models to further boost effective interactive medical image segmentation. The code will be released at \url{https://github.com/goodtime-123/LIM-Net}.