Transfer learning with class activation maps in compositions driving plaque classification in carotid ultrasound

TL;DR

This paper explores how CNNs classify carotid ultrasound images of plaques using class activation maps to identify which plaque features influence classification.

Contribution

The study introduces the use of class activation maps to interpret CNN-based classification of asymptomatic and symptomatic carotid plaques in ultrasound images.

Findings

Dark grayscale areas (GS ≤ 25) and juxtaluminal black areas (JBAs) were influential in both asymptomatic and symptomatic plaque classification.

Lipid cores, fibrous content, and calcified areas were more associated with asymptomatic plaques.

The model achieved 80.4% accuracy in classifying plaque images into asymptomatic and symptomatic categories.

Abstract

Carotid B-mode ultrasound (U/S) imaging provides more than the degree of stenosis in stroke risk assessment. Plaque morphology and texture have been extensively investigated in U/S images, revealing plaque components, such as juxtaluminal black areas close to lumen (JBAs), whose size is linearly related to the risk of stroke. Convolutional neural networks (CNNs) have joined the battle for the identification of high-risk plaques, although the ways they perceive asymptomatic (ASY) and symptomatic (SY) plaque features need further investigation. In this study, the objective was to assess whether class activations maps (CAMs) can reveal which U/S grayscale-(GS)-based plaque compositions (lipid cores, fibrous content, collagen, and/or calcified areas) influence the model's understanding of the ASY and SY cases.

We used Xception via transfer learning, as a base for feature extraction (all layers frozen), whose output we fed into a new dense layer, followed by a new classification layer, which we trained with standardized B-mode U/S longitudinal plaque images. From a total of 236 images (118 ASY and 118 SY), we used 168 in training (84 ASY and 84 SY), 22 in internal validation (11 ASY and 11 SY), and 46 in testing (23 ASY and 23 SY).

In testing, the model reached an accuracy, sensitivity, specificity, and area under the curve at 80.4%, 82.6%, 78.3%, and 0.80, respectively. Precision and the F1 score were found at 81.8% and 80.0%, and 79.2% and 80.9%, for the ASY and SY cases, respectively. We used faster-Score-CAM to produce a heatmap for each tested image, quantifying each plaque composition area overlapping with the heatmap to find compositions areas related to ASY and SY cases. Dark areas (GS ≤ 25) or JBAs (whose presence was verified priorly, by an experienced vascular surgeon) were found influential for the understanding of both the ASY and the SY plaques. Calcified areas, fibrous content, and lipid cores, together, were more related to ASY plaques.

These findings indicate the need for further investigation on how the GS ≤ 25 plaque areas affect the learning process of the CNN models, and they will be further validated.

Figures