ICH-SCNet: Intracerebral Hemorrhage Segmentation and Prognosis Classification Network Using CLIP-guided SAM mechanism

TL;DR

This paper presents ICH-SCNet, a multi-task neural network that leverages CLIP-guided SAM mechanism to improve intracerebral hemorrhage segmentation and prognosis classification by integrating imaging and text data.

Contribution

The novel ICH-SCNet model combines cross-modal interaction and multi-task learning for simultaneous ICH segmentation and prognosis prediction, outperforming existing methods.

Findings

Outperforms state-of-the-art in segmentation metrics

Achieves higher accuracy in prognosis classification

Effectively integrates multimodal data for improved results

Abstract

Intracerebral hemorrhage (ICH) is the most fatal subtype of stroke and is characterized by a high incidence of disability. Accurate segmentation of the ICH region and prognosis prediction are critically important for developing and refining treatment plans for post-ICH patients. However, existing approaches address these two tasks independently and predominantly focus on imaging data alone, thereby neglecting the intrinsic correlation between the tasks and modalities. This paper introduces a multi-task network, ICH-SCNet, designed for both ICH segmentation and prognosis classification. Specifically, we integrate a SAM-CLIP cross-modal interaction mechanism that combines medical text and segmentation auxiliary information with neuroimaging data to enhance cross-modal feature recognition. Additionally, we develop an effective feature fusion module and a multi-task loss function to improve performance further. Extensive experiments on an ICH dataset reveal that our approach surpasses other state-of-the-art methods. It excels in the overall performance of classification tasks and outperforms competing models in all segmentation task metrics.