Large-scale modality-invariant foundation models for brain MRI analysis: Application to lesion segmentation

TL;DR

This paper introduces a large-scale, modality-invariant foundation model for brain MRI analysis that improves lesion segmentation by learning anatomical priors from unlabeled data, highlighting the importance of modality-specific features.

Contribution

It proposes a novel modality-invariant representation learning framework tailored for multi-modal MRI, enhancing lesion segmentation performance in neuroimaging tasks.

Findings

Cross-modality alignment achieved successfully.

Lesion segmentation benefits from modality-specific features.

Model and code are publicly available.

Abstract

The field of computer vision is undergoing a paradigm shift toward large-scale foundation model pre-training via self-supervised learning (SSL). Leveraging large volumes of unlabeled brain MRI data, such models can learn anatomical priors that improve few-shot performance in diverse neuroimaging tasks. However, most SSL frameworks are tailored to natural images, and their adaptation to capture multi-modal MRI information remains underexplored. This work proposes a modality-invariant representation learning setup and evaluates its effectiveness in stroke and epilepsy lesion segmentation, following large-scale pre-training. Experimental results suggest that despite successful cross-modality alignment, lesion segmentation primarily benefits from preserving fine-grained modality-specific features. Model checkpoints and code are made publicly available.