Multi-Sequence Parotid Gland Lesion Segmentation via Expert Text-Guided Segment Anything Model

TL;DR

This paper introduces PG-SAM, a novel expert text-guided segmentation model for parotid gland lesions that leverages domain knowledge and multi-sequence imaging to improve accuracy in clinical applications.

Contribution

The study presents a new expert diagnosis text-guided SAM model that incorporates domain knowledge and cross-sequence attention for improved lesion segmentation.

Findings

Achieves state-of-the-art performance across three clinical centers.

Effectively integrates expert domain knowledge via diagnosis reports.

Enhances segmentation accuracy with multi-sequence attention modules.

Abstract

Parotid gland lesion segmentation is essential for the treatment of parotid gland diseases. However, due to the variable size and complex lesion boundaries, accurate parotid gland lesion segmentation remains challenging. Recently, the Segment Anything Model (SAM) fine-tuning has shown remarkable performance in the field of medical image segmentation. Nevertheless, SAM's interaction segmentation model relies heavily on precise lesion prompts (points, boxes, masks, etc.), which are very difficult to obtain in real-world applications. Besides, current medical image segmentation methods are automatically generated, ignoring the domain knowledge of medical experts when performing segmentation. To address these limitations, we propose the parotid gland segment anything model (PG-SAM), an expert diagnosis text-guided SAM incorporating expert domain knowledge for cross-sequence parotid gland lesion segmentation. Specifically, we first propose an expert diagnosis report guided prompt generation module that can automatically generate prompt information containing the prior domain knowledge to guide the subsequent lesion segmentation process. Then, we introduce a cross-sequence attention module, which integrates the complementary information of different modalities to enhance the segmentation effect. Finally, the multi-sequence image features and generated prompts are feed into the decoder to get segmentation result. Experimental results demonstrate that PG-SAM achieves state-of-the-art performance in parotid gland lesion segmentation across three independent clinical centers, validating its clinical applicability and the effectiveness of diagnostic text for enhancing image segmentation in real-world clinical settings.