# ProMMF_Kron: a multimodal deep learning model for immunotherapy response prediction in stomach adenocarcinoma

## TL;DR

This paper introduces ProMMF_Kron, a deep learning model that improves prediction of immunotherapy response in stomach cancer by combining molecular and image data.

## Contribution

The novel two-stage feature fusion strategy using Kronecker product and back-projection modules enhances multimodal prediction of MSI/MSS subtypes.

## Key findings

- ProMMF_Kron achieved an AUC of 0.96 in distinguishing MSI and MSS subtypes.
- The model outperformed single-modality and other multimodal approaches by 3.2% and 4.3% in AUC, respectively.
- The model showed high generalization capability on colorectal cancer datasets.

## Abstract

Immune checkpoint inhibitor (ICI) therapy has significantly improved treatment outcomes for various cancers by enhancing T cell-mediated anti-tumor immune responses. However, accurately predicting patient response to ICI treatment remains a major challenge due to the risk of immune-related adverse events. Microsatellite instability (MSI), as an important molecular biomarker characterized by high mutation rates and abundant tumor neoantigen production, has been demonstrated to effectively predict clinical benefits from immunotherapy. In gastric adenocarcinoma (STAD) patients, approximately 22% exhibit the MSI subtype while the majority are microsatellite stable (MSS). This significant molecular heterogeneity underscores the urgent need to develop reliable predictive tools.

To address this problem, we developed a multimodal deep learning model named ProMMF_Kron based on a multicenter dataset comprising 282 patients. The model employs a two stage feature fusion strategy: first extracting key features from both molecular profiles and pathological images through differential gene analysis and a pretrained deep convolutional neural network, respectively; then designing a sophisticated fusion architecture incorporating Kronecker product operations and back-projection modules to achieve efficient interaction between gene expression features and pathological image features. The dataset was partitioned into training, validation, and testing sets at a ratio of 6:2:2.

Experimental results demonstrate that the ProMMF_Kron model effectively distinguishes between MSI and MSS subtypes (MSI versus MSS) and exhibits competitive predictive performance on independent test datasets, achieving an AUC of 0.96 (95% CI: 0.89-1.00), outperforming traditional single-modality prediction models (3.2% AUC improvement) and other multimodal fusion approaches (4.3% AUC improvement). Further validation confirms the model’s excellent stability and generalization capability, maintaining high predictive accuracy on colorectal cancer (CRC) dataset.

Through bioinformatics analysis and feature visualization techniques, this study also reveals potential associations between key molecular biomarkers and critical immune regulatory pathways, providing a powerful decision-support tool for precision immunotherapy in gastric cancer with substantial clinical translation value and application prospects.