# PI-MMNet: a cross-modal neural network for predicting neurological deterioration in pontine infarction

## TL;DR

PI-MMNet is a new neural network that predicts neurological deterioration in pontine infarction by combining MRI and clinical data more efficiently than existing methods.

## Contribution

PI-MMNet introduces a novel cross-modal neural network with adaptive fusion and graph modules for efficient and accurate prediction of neurological deterioration.

## Key findings

- PI-MMNet outperformed existing methods with improvements in accuracy, F1, and AUC.
- The model achieved better performance using fewer parameters and memory compared to the strongest baseline.
- Ablation studies confirmed the effectiveness of each module in the network.

## Abstract

Pontine infarction, a subtype of ischemic stroke, often leads to neurological deterioration (ND). Current diagnostic methods rely mainly on imaging and neglect clinical data, while existing multimodal models struggle with small lesions, heterogeneous inputs, and high computational cost.

We propose PI-MMNet, a cross-modal neural network combining: (i) a Multi-modal Feature Processing module with Mamba-based extractors, (ii) a Dynamic Residual Fusion module for robust feature integration, and (iii) an Adaptive Graph module for efficient relational reasoning. A multi-loss strategy jointly optimizes alignment, graph consistency, and classification. Experiments used 386 pontine infarction cases with MRI and clinical data under 5-fold cross-validation.

PI-MMNet outperformed state-of-the-art methods, improving accuracy by 1.03%, F1 by 0.0504, and AUC by 0.0343, while using only 146 parameters and 135 memory of the strongest baseline. Ablation and visualization confirmed the contributions of all modules.

PI-MMNet provides an efficient and interpretable framework for predicting ND in pontine infarction and may generalize to other multimodal medical tasks. Our code is available at https://github.com/jinhui66/PI-MMNet.