Towards Scalable and Robust White Matter Lesion Localization via Multimodal Deep Learning

TL;DR

This paper presents a deep learning framework for white matter lesion segmentation and localization using multimodal MRI, improving accuracy and robustness, and exploring multi-task joint prediction versus separate models.

Contribution

It introduces a flexible multimodal deep learning approach that handles missing modalities and evaluates joint versus separate lesion and region prediction models.

Findings

Multimodal MRI significantly improves segmentation accuracy.

Modality-interchangeable setup enhances robustness with some accuracy trade-off.

Joint lesion-region segmentation was less effective than separate models.

Abstract

White matter hyperintensities (WMH) are radiological markers of small vessel disease and neurodegeneration, whose accurate segmentation and spatial localization are crucial for diagnosis and monitoring. While multimodal MRI offers complementary contrasts for detecting and contextualizing WM lesions, existing approaches often lack flexibility in handling missing modalities and fail to integrate anatomical localization efficiently. We propose a deep learning framework for WM lesion segmentation and localization that operates directly in native space using single- and multi-modal MRI inputs. Our study evaluates four input configurations: FLAIR-only, T1-only, concatenated FLAIR and T1, and a modality-interchangeable setup. It further introduces a multi-task model for jointly predicting lesion and anatomical region masks to estimate region-wise lesion burden. Experiments conducted on the MICCAI WMH Segmentation Challenge dataset demonstrate that multimodal input significantly improves the segmentation performance, outperforming unimodal models. While the modality-interchangeable setting trades accuracy for robustness, it enables inference in cases with missing modalities. Joint lesion-region segmentation using multi-task learning was less effective than separate models, suggesting representational conflict between tasks. Our findings highlight the utility of multimodal fusion for accurate and robust WMH analysis, and the potential of joint modeling for integrated predictions.