PIMMS: Permutation Invariant Multi-Modal Segmentation

TL;DR

PIMMS is a novel neural network method for multi-modal MRI segmentation that operates without modality labels, demonstrating competitive or superior performance to label-dependent models in clinical settings.

Contribution

Introduces PIMMS, a permutation invariant multi-modal segmentation approach that handles unlabeled MRI data, addressing variability in clinical imaging protocols.

Findings

Outperforms baseline models with modality labels in some settings

Achieves comparable performance to label-dependent models

Operates effectively without modality labels in MRI segmentation

Abstract

In a research context, image acquisition will often involve a pre-defined static protocol and the data will be of high quality. If we are to build applications that work in hospitals without significant operational changes in care delivery, algorithms should be designed to cope with the available data in the best possible way. In a clinical environment, imaging protocols are highly flexible, with MRI sequences commonly missing appropriate sequence labeling (e.g. T1, T2, FLAIR). To this end we introduce PIMMS, a Permutation Invariant Multi-Modal Segmentation technique that is able to perform inference over sets of MRI scans without using modality labels. We present results which show that our convolutional neural network can, in some settings, outperform a baseline model which utilizes modality labels, and achieve comparable performance otherwise.