Does Interrater Variation Influence the Classification According to the Diagnostic Criteria for Multiple Sclerosis?

TL;DR

This study examines how much MRI analysis of brain, optic nerve, and spinal cord lesions varies between raters and how this affects multiple sclerosis diagnosis using updated criteria.

Contribution

The study evaluates interrater agreement in MRI lesion detection and its impact on MS diagnosis using the latest McDonald criteria.

Findings

Interrater agreement for lesion presence was moderate to substantial, with highest agreement for spinal cord lesions.

Diagnostic classification agreement remained high between the 2017 and 2024 McDonald criteria revisions.

Despite standardized imaging, lesion count agreement among raters was limited.

Abstract

Studies on interrater agreement in MRI analysis for patients experiencing a first clinical episode suggestive of multiple sclerosis (MS) are limited and outdated. This study aimed to evaluate interrater agreement for lesion identification in the brain, optic nerve, and spinal cord, and to assess variability in applying the 2017 and 2024 McDonald criteria.

Seventy‐eight patients underwent a standardized multisequence 3 Tesla MRI of the brain, optic nerves, and spinal cord, analyzed by three readers with varying expertise. Lesions were categorized based on location according to McDonald criteria. Interrater variability was measured using Cohen's κ for pairwise reader comparisons and Fleiss κ for overall agreement.

Interrater agreement ranged from slight to fair for total lesion count and moderate to substantial for lesion presence. The highest agreement occurred for spinal cord lesions (κ = 0.84), periventricular T2 lesions (κ = 0.70), and gadolinium‐enhancing brain lesions (κ = 0.75). Classification agreement based on diagnostic criteria was substantial between the 2017 and 2024 McDonald criteria revisions (κ = 0.65).

Despite advanced standardized imaging protocols at 3 T, interrater agreement regarding lesion counts does not improve substantially. However, agreement in classifying lesions according to diagnostic criteria is consistent between the 2017 and 2024 criteria.

This study investigates interrater variability in MRI‐based lesion detection and its impact on multiple sclerosis diagnosis in accordance with the 2017 and 2024 McDonald criteria. Despite differences among raters, diagnostic agreement between both criteria versions remained high. The results show that diagnostic consistency is strong and that standardized MRI interpretation is important.

Figures