# Explainable multi-modal deep learning for transparent cancer diagnosis: integrating radiology, clinical features, and decision visualization

## TL;DR

This paper introduces a transparent AI framework that combines medical imaging and clinical data to improve cancer diagnosis accuracy and trust.

## Contribution

The novel framework integrates multi-modal data with attention-based fusion and provides cross-modal explanations using Grad-CAM++ and SHAP.

## Key findings

- The model outperformed uni-modal approaches and simple fusion baselines in diagnostic performance.
- Attention-based fusion showed better results than feature concatenation without sacrificing accuracy.
- Explanations highlighted relevant tumor regions and clinical risk factors, showing strong generalization across datasets.

## Abstract

Although artificial intelligence–based cancer diagnostic models have demonstrated strong predictive performance, their lack of transparency and reliance on single-modality data continue to limit clinical trust and adoption. Effectively integrating multi-modal data with interpret-able decision-making remains a key challenge.

We propose an explainable multi-modal deep learning framework that integrates radiological imaging and structured clinical features using attention-based fusion. Image-level explanations are generated using Grad-CAM++, while SHAP is employed to quantify clinical feature contributions, enabling unified and cross-modal aligned interpretation rather than independent uni-modal explanations. The framework was evaluated on publicly available datasets, including CBIS-DDSM mammography, Duke Breast Cancer MRI, and TCGA cohorts (BRCA, LUAD, and GBM), comprising a total of 3,842 images from 2,917 patients.

The proposed model consistently outperformed uni-modal approaches and simple fusion baselines, achieving an improved balance between sensitivity and specificity. Attention-based fusion demonstrated superior performance compared with feature concatenation, and the integration of explainability did not compromise predictive accuracy. Visual and clinical explanations highlighted diagnostically relevant tumor regions and established oncological risk factors. Stable performance across datasets indicates strong generalization capability.

These results demonstrate that explainable multi-modal learning can effectively combine accuracy, interpret-ability, and robustness, supporting the development of reliable AI-based decision-support systems for cancer diagnosis.