# Multimodal fusion of pathology and radiology foundation models for WHO 2021 glioma subtyping

## TL;DR

This paper introduces a multimodal framework combining histopathology and MRI data to improve glioma subtyping, achieving strong performance using unpaired datasets.

## Contribution

A novel multimodal fusion framework using foundation models for glioma subtyping without requiring paired data.

## Key findings

- Multimodal models outperform unimodal models in glioma subtyping accuracy.
- A mixture-of-experts architecture achieves the highest performance with an AUC of 0.98 on validation data.
- The model identifies interpretable visual biomarkers linked to glioma subtypes.

## Abstract

Molecular subtyping of gliomas is a common clinical task, yet challenging to perform on histology or radiology images alone. To address this challenge, we developed a multimodal classification framework that integrates histopathology and magnetic resonance imaging (MRI) using foundation models as unimodal experts, and evaluated three modality fusion strategies. Models are trained on two unpaired datasets of 772 histopathology cases and 959 multiparametric MRI scans, and tested on 171 unseen patient-matched cases. Multimodal models consistently outperform their unimodal counterparts, with a mixture-of-experts architecture achieving the strongest performance (AUC = 0.98 validation; AUC = 0.94 independent test set). Notably, we show that high-performing multimodal classifiers can be trained even without paired multimodal data, and in purely unimodal settings, these models match unimodal baselines. Finally, a detailed analysis of the learned multimodal representations reveals that the model identifies distinct visual biomarkers associated with glioma molecular subtypes, providing interpretable insight into its decision-making process.