MMFusion: Multi-modality Diffusion Model for Lymph Node Metastasis Diagnosis in Esophageal Cancer

TL;DR

This paper introduces MMFusion, a multi-modal diffusion model utilizing heterogeneous graphs and relational learning to improve lymph node metastasis diagnosis in esophageal cancer from CT, clinical, and radiomics data.

Contribution

It proposes a novel multi-modal heterogeneous graph-based diffusion model with relational learning for better multi-modal data integration and prognostic correlation in cancer diagnosis.

Findings

Effective elimination of information redundancy

Improved diagnostic accuracy demonstrated in experiments

Uncovering latent prognostic relationships

Abstract

Esophageal cancer is one of the most common types of cancer worldwide and ranks sixth in cancer-related mortality. Accurate computer-assisted diagnosis of cancer progression can help physicians effectively customize personalized treatment plans. Currently, CT-based cancer diagnosis methods have received much attention for their comprehensive ability to examine patients' conditions. However, multi-modal based methods may likely introduce information redundancy, leading to underperformance. In addition, efficient and effective interactions between multi-modal representations need to be further explored, lacking insightful exploration of prognostic correlation in multi-modality features. In this work, we introduce a multi-modal heterogeneous graph-based conditional feature-guided diffusion model for lymph node metastasis diagnosis based on CT images as well as clinical measurements and radiomics data. To explore the intricate relationships between multi-modal features, we construct a heterogeneous graph. Following this, a conditional feature-guided diffusion approach is applied to eliminate information redundancy. Moreover, we propose a masked relational representation learning strategy, aiming to uncover the latent prognostic correlations and priorities of primary tumor and lymph node image representations. Various experimental results validate the effectiveness of our proposed method. The code is available at https://github.com/wuchengyu123/MMFusion.