Co-Attentive Cross-Modal Deep Learning for Medical Evidence Synthesis and Decision Making

TL;DR

This paper introduces a novel cross-modal deep learning architecture with co-attention for efficient and accurate medical evidence synthesis, demonstrated by improved Parkinson's Disease diagnosis and insights into biological pathways.

Contribution

It presents a new co-attentive cross-modal deep learning model that models complex relationships between multimodal data, improving diagnosis accuracy and efficiency in medical decision making.

Findings

Outperforms state-of-the-art unimodal analysis by 2.35%.

Reduces model parameters by 53% compared to industry standard.

Identifies a novel biological link involving interferon-gamma, DNA methylation, and Parkinson's Disease.

Abstract

Modern medicine requires generalised approaches to the synthesis and integration of multimodal data, often at different biological scales, that can be applied to a variety of evidence structures, such as complex disease analyses and epidemiological models. However, current methods are either slow and expensive, or ineffective due to the inability to model the complex relationships between data modes which differ in scale and format. We address these issues by proposing a cross-modal deep learning architecture and co-attention mechanism to accurately model the relationships between the different data modes, while further reducing patient diagnosis time. Differentiating Parkinson's Disease (PD) patients from healthy patients forms the basis of the evaluation. The model outperforms the previous state-of-the-art unimodal analysis by 2.35%, while also being 53% more parameter efficient than the industry standard cross-modal model. Furthermore, the evaluation of the attention coefficients allows for qualitative insights to be obtained. Through the coupling with bioinformatics, a novel link between the interferon-gamma-mediated pathway, DNA methylation and PD was identified. We believe that our approach is general and could optimise the process of medical evidence synthesis and decision making in an actionable way.