Machine learning-based combination of the central vein sign, cortical lesions and paramagnetic rim lesions: a web-based tool for the diagnosis of multiple sclerosis

TL;DR

This study shows that combining specific MRI biomarkers with machine learning improves the accuracy of diagnosing multiple sclerosis, and a web tool is available for clinical use.

Contribution

The novel contribution is a machine learning-based diagnostic framework using simplified MRI biomarkers that outperforms existing criteria for multiple sclerosis.

Findings

51 machine learning models outperformed the baseline McDonald criteria with up to 13% higher balanced accuracy.

A simplified logistic regression model achieved 94.7% balanced accuracy without significant difference from the best full-count model.

External validation confirmed robust performance, with the simplified model reaching 97.2% balanced accuracy on an out-of-distribution test set.

Abstract

Multiple sclerosis diagnostic criteria lack optimal specificity, leading to potential misdiagnosis. Advanced magnetic resonance imaging (MRI) biomarkers like the central vein sign, cortical lesions and paramagnetic rim lesions are highly specific to multiple sclerosis and could potentially improve diagnostic accuracy. In this study, we applied machine learning techniques to a retrospective, multicentric dataset of 322 multiple sclerosis/multiple sclerosis-mimic (204/118) and 84 prodromal multiple sclerosis/non-multiple sclerosis (43/41) adult patients, incorporating the central vein sign, cortical lesions and paramagnetic rim lesions. We compared (5 × 2 cross-validation combined F-test) the diagnostic performance of 71 machine learning models, each corresponding to a distinct combination of full-count or simplified biomarker inputs, against the baseline dissemination in space McDonald criteria. The aim was to evaluate the multiple sclerosis diagnostic power of combining these biomarkers in an MRI-only diagnostic framework. 51 of the 71 models significantly outperformed the dissemination in space criterion (P < 0.05), with balanced accuracy improvements up to 13.0% (confidence interval: [+10.5; +17.0]). The best overall model (random forest, using full-count assessments) achieved 95.7% (confidence interval: [93.2; 99.7]) balanced accuracy; the best simplified model (logistic regression, using only simplified assessments) reached 94.7% with no significant difference with the former (P = 0.29). Notably, 12/51 high-performing models used only simplified assessments. To further investigate the models’ generalizability, external validation on two out-of-distribution test sets using bootstrapping (1000 resamples) confirmed these results and highlighted a more robust generalization for the best model using solely simplified biomarkers. On the first external test set (n = 37, Verona), the simplified model achieved 97.2% balanced accuracy, while the full-count model reached 93.3% (versus 83.3% for baseline). On the second test set (n = 84, prodromal cases), the simplified model achieved 92.6% (versus 60.1% for baseline) showing competitive performance against the full-count model (93.9%). Both models improved all key performance metrics—balanced accuracy, sensitivity, specificity, precision and F1 score—over the baseline on both test sets (all P < 0.0001). Within a non-invasive MRI-only diagnostic framework, these results show that the incorporation of advanced imaging biomarkers into the multiple sclerosis-MRI diagnostic criteria significantly enhances the diagnostic accuracy—a statement holding true even when using simplified central vein sign, cortical lesions and paramagnetic rim lesions assessments. The study also provides a publicly available online diagnostic tool, facilitating further interaction, validation and clinical support (https://www.msdiagnostictool.org).

Wynen, Vanden Bulcke et al. report that machine learning models incorporating advanced imaging biomarkers—central vein sign, cortical lesions and paramagnetic rim lesions—improve multiple sclerosis -MRI diagnostic accuracy, with improved robustness observed in models using clinically-intuitive simplified biomarker assessment. The authors published an online tool enabling interaction with trained models.

Graphical AbstractFor image description, please refer to the figure legend and surrounding text.

Figures