Architecture-Agnostic Modality-Isolated Gated Fusion for Robust Multi-Modal Prostate MRI Segmentation

TL;DR

This paper introduces MIGF, a modality-isolated gating fusion module that enhances robustness in multi-modal prostate MRI segmentation, especially under missing or degraded diffusion sequences, across various architectures.

Contribution

The proposed MIGF module is architecture-agnostic, improves robustness to missing or artifacted modalities, and includes modality dropout training for better compensation.

Findings

MIGF improved ideal-scenario Ranking Score by up to 13.4%.

Robustness gains mainly from modality isolation and dropout-driven compensation.

External evaluation revealed domain shift due to ADC map incompatibility.

Abstract

Multi-parametric prostate MRI combines T2-weighted (T2W), apparent diffusion coefficient (ADC), and high b-value diffusion-weighted (HBV) sequences for non-invasive detection of clinically significant prostate cancer. In practice, the diffusion sequences are more frequently subject to acquisition variability, motion, and artifacts than T2W, making robust fusion of these channels the clinically relevant requirement. We propose Modality-Isolated Gated Fusion (MIGF), an architecture-agnostic module that maintains separate modality-specific encoding streams before a learned gating stage, combined with modality dropout training to enforce compensation under incomplete inputs. We benchmark six backbones and assess MIGF-equipped models under seven missing-modality and artifact scenarios on PI-CAI (1,500 studies, fold-0 split, five seeds). MIGF improved ideal-scenario Ranking Score for UNet, nnUNet, and Mamba by 2.8%, 4.6%, and 13.4%; the best model (MIGFNet-nnUNet, gating + ModDrop, no deep supervision) achieved 0.7304 +/- 0.056. MIGF primarily improved tolerance to HBV/ADC degradation, while missing T2W remained a shared failure mode across all architectures. Mechanistic analysis shows that robustness gains arise from strict modality isolation and dropout-driven compensation rather than adaptive per-sample quality routing: the gate converged to a stable modality prior, and deep supervision was beneficial only for the largest backbone. External evaluation on Prostate158 (n=158) revealed substantial domain shift driven primarily by ADC map incompatibility across institutions; modality-isolation analysis confirmed that removing ADC improved external performance, identifying computed diffusion maps as inherently less portable than raw-acquired sequences.