Feasibility and benefits of joint learning from MRI databases with different brain diseases and modalities for segmentation

TL;DR

This study demonstrates that joint learning from multiple multi-modal MRI databases with different brain pathologies is feasible and beneficial, enabling a single model to segment various pathologies and modalities, and adapt to new data.

Contribution

It introduces methods for joint training across diverse MRI databases, showing practical benefits and enabling segmentation of multiple pathologies and modalities with a single model.

Findings

Joint training is feasible across multiple MRI databases with different pathologies.

A single model can effectively segment diverse pathologies and modalities.

Joint learning facilitates adaptation to new datasets and pathologies.

Abstract

Models for segmentation of brain lesions in multi-modal MRI are commonly trained for a specific pathology using a single database with a predefined set of MRI modalities, determined by a protocol for the specific disease. This work explores the following open questions: Is it feasible to train a model using multiple databases that contain varying sets of MRI modalities and annotations for different brain pathologies? Will this joint learning benefit performance on the sets of modalities and pathologies available during training? Will it enable analysis of new databases with different sets of modalities and pathologies? We develop and compare different methods and show that promising results can be achieved with appropriate, simple and practical alterations to the model and training framework. We experiment with 7 databases containing 5 types of brain pathologies and different sets of MRI modalities. Results demonstrate, for the first time, that joint training on multi-modal MRI databases with different brain pathologies and sets of modalities is feasible and offers practical benefits. It enables a single model to segment pathologies encountered during training in diverse sets of modalities, while facilitating segmentation of new types of pathologies such as via follow-up fine-tuning. The insights this study provides into the potential and limitations of this paradigm should prove useful for guiding future advances in the direction. Code and pretrained models: https://github.com/WenTXuL/MultiUnet