Dual-Modal Lung Cancer AI: Interpretable Radiology and Microscopy with Clinical Risk Integration

TL;DR

This study introduces a dual-modal AI system combining CT radiology and histopathology for lung cancer diagnosis, achieving high accuracy and interpretability through explainable AI techniques.

Contribution

The paper presents a novel multimodal framework integrating radiology and histopathology with clinical data, enhancing diagnostic accuracy and interpretability in lung cancer classification.

Findings

Accuracy up to 0.87 in classification

AUROC above 0.97 indicating strong discriminative ability

Grad-CAM++ provided the best interpretability and localization

Abstract

Lung cancer remains one of the leading causes of cancer-related mortality worldwide. Conventional computed tomography (CT) imaging, while essential for detection and staging, has limitations in distinguishing benign from malignant lesions and providing interpretable diagnostic insights. To address this challenge, this study proposes a dual-modal artificial intelligence framework that integrates CT radiology with hematoxylin and eosin (H&E) histopathology for lung cancer diagnosis and subtype classification. The system employs convolutional neural networks to extract radiologic and histopathologic features and incorporates clinical metadata to improve robustness. Predictions from both modalities are fused using a weighted decision-level integration mechanism to classify adenocarcinoma, squamous cell carcinoma, large cell carcinoma, small cell lung cancer, and normal tissue. Explainable AI techniques including Grad-CAM, Grad-CAM++, Integrated Gradients, Occlusion, Saliency Maps, and SmoothGrad are applied to provide visual interpretability. Experimental results show strong performance with accuracy up to 0.87, AUROC above 0.97, and macro F1-score of 0.88. Grad-CAM++ achieved the highest faithfulness and localization accuracy, demonstrating strong correspondence with expert-annotated tumor regions. These results indicate that multimodal fusion of radiology and histopathology can improve diagnostic performance while maintaining model transparency, suggesting potential for future clinical decision support systems in precision oncology.