Fusing Medical Image Features and Clinical Features with Deep Learning for Computer-Aided Diagnosis

TL;DR

This paper introduces a deep learning approach that fuses medical imaging features with clinical data to enhance diagnostic accuracy across various medical conditions.

Contribution

It presents a novel neural network architecture that integrates image and clinical features, with clinical data guiding image feature extraction for improved diagnosis.

Findings

Improved diagnostic accuracy in Alzheimer's disease detection

Effective fusion of imaging and clinical data enhances classification stability

Clinical-guided feature extraction outperforms traditional methods

Abstract

Current Computer-Aided Diagnosis (CAD) methods mainly depend on medical images. The clinical information, which usually needs to be considered in practical clinical diagnosis, has not been fully employed in CAD. In this paper, we propose a novel deep learning-based method for fusing Magnetic Resonance Imaging (MRI)/Computed Tomography (CT) images and clinical information for diagnostic tasks. Two paths of neural layers are performed to extract image features and clinical features, respectively, and at the same time clinical features are employed as the attention to guide the extraction of image features. Finally, these two modalities of features are concatenated to make decisions. We evaluate the proposed method on its applications to Alzheimer's disease diagnosis, mild cognitive impairment converter prediction and hepatic microvascular invasion diagnosis. The encouraging experimental results prove the values of the image feature extraction guided by clinical features and the concatenation of two modalities of features for classification, which improve the performance of diagnosis effectively and stably.