Multi-modal Intermediate Feature Interaction AutoEncoder for Overall Survival Prediction of Esophageal Squamous Cell Cancer

TL;DR

This paper introduces a novel autoencoder-based deep learning model that enhances multi-modal feature interaction and alignment for improved overall survival prediction in esophageal squamous cell cancer.

Contribution

It proposes new modules for multi-modal feature reinforcement and a joint loss for better feature alignment, addressing previous limitations in multi-modal survival prediction.

Findings

Achieved improved discriminative ability in survival prediction.

Enhanced risk stratification accuracy.

Validated effectiveness of proposed modules through experiments.

Abstract

Survival prediction for esophageal squamous cell cancer (ESCC) is crucial for doctors to assess a patient's condition and tailor treatment plans. The application and development of multi-modal deep learning in this field have attracted attention in recent years. However, the prognostically relevant features between cross-modalities have not been further explored in previous studies, which could hinder the performance of the model. Furthermore, the inherent semantic gap between different modal feature representations is also ignored. In this work, we propose a novel autoencoder-based deep learning model to predict the overall survival of the ESCC. Two novel modules were designed for multi-modal prognosis-related feature reinforcement and modeling ability enhancement. In addition, a novel joint loss was proposed to make the multi-modal feature representations more aligned. Comparison and ablation experiments demonstrated that our model can achieve satisfactory results in terms of discriminative ability, risk stratification, and the effectiveness of the proposed modules.