Adaptive Interactive Segmentation for Multimodal Medical Imaging via Selection Engine

TL;DR

This paper introduces SISeg, an adaptive interactive segmentation model for multimodal medical imaging that leverages a selection engine and an automated frame selection system to improve accuracy and generalization across diverse modalities.

Contribution

The paper presents a novel strategy-driven segmentation model with an adaptive frame selection engine, enhancing multi-modal medical image segmentation without extensive prior knowledge.

Findings

Demonstrated robust adaptability across 10 datasets and 7 modalities.

Improved segmentation accuracy and generalization in multi-modal tasks.

Enhanced interpretability through interactive feedback mechanism.

Abstract

In medical image analysis, achieving fast, efficient, and accurate segmentation is essential for automated diagnosis and treatment. Although recent advancements in deep learning have significantly improved segmentation accuracy, current models often face challenges in adaptability and generalization, particularly when processing multi-modal medical imaging data. These limitations stem from the substantial variations between imaging modalities and the inherent complexity of medical data. To address these challenges, we propose the Strategy-driven Interactive Segmentation Model (SISeg), built on SAM2, which enhances segmentation performance across various medical imaging modalities by integrating a selection engine. To mitigate memory bottlenecks and optimize prompt frame selection during the inference of 2D image sequences, we developed an automated system, the Adaptive Frame Selection Engine (AFSE). This system dynamically selects the optimal prompt frames without requiring extensive prior medical knowledge and enhances the interpretability of the model's inference process through an interactive feedback mechanism. We conducted extensive experiments on 10 datasets covering 7 representative medical imaging modalities, demonstrating the SISeg model's robust adaptability and generalization in multi-modal tasks. The project page and code will be available at: [URL].