An Arbitrary-Modal Fusion Network for Volumetric Cranial Nerves Tract Segmentation

TL;DR

This paper introduces CNTSeg-v2, a versatile neural network capable of segmenting cranial nerve tracts from various MRI modality combinations, improving accuracy and practicality in clinical settings.

Contribution

The novel arbitrary-modal fusion network handles different modality combinations with a primary T1w supervision and a specialized decoder, advancing cranial nerve segmentation methods.

Findings

Achieves state-of-the-art segmentation performance.

Outperforms existing methods on HCP and MDM datasets.

Effectively handles incomplete multimodal data.

Abstract

The segmentation of cranial nerves (CNs) tract provides a valuable quantitative tool for the analysis of the morphology and trajectory of individual CNs. Multimodal CNs tract segmentation networks, e.g., CNTSeg, which combine structural Magnetic Resonance Imaging (MRI) and diffusion MRI, have achieved promising segmentation performance. However, it is laborious or even infeasible to collect complete multimodal data in clinical practice due to limitations in equipment, user privacy, and working conditions. In this work, we propose a novel arbitrary-modal fusion network for volumetric CNs tract segmentation, called CNTSeg-v2, which trains one model to handle different combinations of available modalities. Instead of directly combining all the modalities, we select T1-weighted (T1w) images as the primary modality due to its simplicity in data acquisition and contribution most to the results, which supervises the information selection of other auxiliary modalities. Our model encompasses an Arbitrary-Modal Collaboration Module (ACM) designed to effectively extract informative features from other auxiliary modalities, guided by the supervision of T1w images. Meanwhile, we construct a Deep Distance-guided Multi-stage (DDM) decoder to correct small errors and discontinuities through signed distance maps to improve segmentation accuracy. We evaluate our CNTSeg-v2 on the Human Connectome Project (HCP) dataset and the clinical Multi-shell Diffusion MRI (MDM) dataset. Extensive experimental results show that our CNTSeg-v2 achieves state-of-the-art segmentation performance, outperforming all competing methods.