# Radiosensitivity Prediction of Tumor Patient Based on Deep Fusion of Pathological Images and Genomics

## TL;DR

This paper introduces a new deep learning model that combines genomic data and histopathological images to better predict how cancer patients will respond to radiotherapy.

## Contribution

The novel Resfusion framework integrates gene expression, clinical data, and histopathological images for improved radiosensitivity prediction.

## Key findings

- Resfusion achieved 76.83% AUC on the BRCA dataset for radiosensitivity prediction.
- The model outperformed unimodal and other multimodal methods in cross-validation tests.
- It reached 79.49% AUC on the HNSC dataset, showing effectiveness across cancer types.

## Abstract

The radiosensitivity of cancer patients determines the efficacy of radiotherapy, and patients with low radiosensitivity cannot benefit from radiotherapy. Therefore, accurately predicting radiosensitivity before treatment is essential for personalized and precise radiotherapy. However, most existing studies rely solely on genomic and clinical features, neglecting the tumor microenvironmental information embedded in histopathological images, which limits prediction accuracy. To address this issue, we propose Resfusion, a deep multimodal fusion framework that integrates patient-level gene expression profiles, clinical records, and histopathological images for tumor radiosensitivity prediction. Specifically, the pre-trained large-scale pathology model is used as an image encoder to extract global representations from whole-slide pathological image. Radiosensitivity-related genes are selected using an autoencoder combined with univariate Cox regression, while clinically relevant variables are manually curated. The three modalities are first concatenated and then refined through a self-attention-based module, which captures inter-feature dependencies within the fused representation and highlights complementary information across modalities. The model was evaluated using five-fold cross-validation on two common tumor datasets suitable for radiotherapy: the Breast Invasive Carcinoma (BRCA) dataset (282 patients in total, with each fold partitioned into 226 training samples and 56 validation samples) and the Head and Neck Squamous Cell Carcinoma (HNSC) dataset (200 patients in total, with each fold partitioned into 161 training samples and 39 validation samples). The average AUC values obtained from the five-fold cross-validation reached 76.83% and 79.49%, respectively. Experimental results demonstrate that the Resfusion model significantly outperforms unimodal methods and existing multimodal fusion methods, verifying its effectiveness in predicting the radiosensitivity of tumor patients.