Multi-modal Medical Image Fusion For Non-Small Cell Lung Cancer Classification

TL;DR

This paper presents a novel multi-modal data fusion approach combining medical images, clinical records, and genomic data with advanced machine learning models to improve non-small cell lung cancer classification accuracy.

Contribution

It introduces a new fusion methodology integrating multi-modal data and leverages state-of-the-art models like MedClip and BEiT for enhanced NSCLC detection.

Findings

Achieved 94.04% accuracy in NSCLC classification

Significant improvements in precision, recall, and F1-score

Enhanced early detection and diagnostic capabilities

Abstract

The early detection and nuanced subtype classification of non-small cell lung cancer (NSCLC), a predominant cause of cancer mortality worldwide, is a critical and complex issue. In this paper, we introduce an innovative integration of multi-modal data, synthesizing fused medical imaging (CT and PET scans) with clinical health records and genomic data. This unique fusion methodology leverages advanced machine learning models, notably MedClip and BEiT, for sophisticated image feature extraction, setting a new standard in computational oncology. Our research surpasses existing approaches, as evidenced by a substantial enhancement in NSCLC detection and classification precision. The results showcase notable improvements across key performance metrics, including accuracy, precision, recall, and F1-score. Specifically, our leading multi-modal classifier model records an impressive accuracy of 94.04%. We believe that our approach has the potential to transform NSCLC diagnostics, facilitating earlier detection and more effective treatment planning and, ultimately, leading to superior patient outcomes in lung cancer care.