# Cascaded Deep Learning-Based Model for Classification and Segmentation of Plaques from Carotid Ultrasound Images

## TL;DR

This paper introduces a deep learning framework that improves classification and segmentation of carotid plaques in ultrasound images to better assess stroke risk.

## Contribution

A novel cascaded framework that shares plaque location information between classification and segmentation tasks using a CAM-guided approach.

## Key findings

- Masked-ResNet-DS achieves a mean F1-score of 96.7% in plaque classification, outperforming competing methods.
- CAM-guided MedSAM achieves a Dice similarity coefficient of 86.6%, surpassing U-Net and nnU-Net.
- Ground truth-based pooling and CAM supervision both improve classification performance.

## Abstract

Carotid plaque classification based on ultrasound echogenicity and quantification of plaque burden are crucial in stroke risk assessment. In this work, we propose a framework that leverages the synergy between classification and segmentation by sharing plaque location information to enhance the performance of both tasks. Our cascaded framework integrates a ResNet-based classifier (Masked-ResNet-DS) with MedSAM, a medically adapted version of the Segment Anything Model for joint classification and segmentation of carotid plaques from 2D ultrasound images. Ground truth boundaries are used to guide region-specific feature pooling in the classifier, helping it focus on plaques during training. Since ground truth boundaries are unavailable at inference, we introduce a two-iteration strategy: the first generates a class activation map (CAM), which is then used for focused pooling in the second iteration to predict plaque type. The CAM is also used as a prompt to guide MedSAM for segmentation. To ensure accurate localization, the CAM is supervised during training using a Dice loss against the segmentation ground truth. Masked-ResNet-DS achieves a mean F1-score of 96.7% in plaque classification, at least 3.2% higher than competing methods. Ablation studies confirm that ground truth-based pooling and CAM supervision both improve classification. CAM-guided MedSAM achieves a Dice similarity coefficient (DSC) of 86.6%, outperforming U-Net and nnU-Net by 5.9% and 3.6%, respectively. In addition, CAM prompts improve MedSAM’s DSC by 2.2%. By sharing plaque location between classification and segmentation, the proposed method improves both tasks and provides a more accurate tool for stroke risk stratification.