Graph Neural Networks with Transformer Fusion of Brain Connectivity Dynamics and Tabular Data for Forecasting Future Tobacco Use

TL;DR

This paper presents GNN-TF, a novel time-aware graph neural network with transformer fusion that effectively integrates brain connectivity and tabular data to improve forecasting of future tobacco use from longitudinal neuroimaging and clinical data.

Contribution

The paper introduces GNN-TF, a new model that combines graph neural networks and transformers to integrate multimodal longitudinal data for outcome prediction.

Findings

GNN-TF outperforms existing models in predicting future tobacco use.

The model effectively captures temporal dynamics in brain connectivity and clinical data.

GNN-TF demonstrates superior accuracy in longitudinal neuroimaging analysis.

Abstract

Integrating non-Euclidean brain imaging data with Euclidean tabular data, such as clinical and demographic information, poses a substantial challenge for medical imaging analysis, particularly in forecasting future outcomes. While machine learning and deep learning techniques have been applied successfully to cross-sectional classification and prediction tasks, effectively forecasting outcomes in longitudinal imaging studies remains challenging. To address this challenge, we introduce a time-aware graph neural network model with transformer fusion (GNN-TF). This model flexibly integrates both tabular data and dynamic brain connectivity data, leveraging the temporal order of these variables within a coherent framework. By incorporating non-Euclidean and Euclidean sources of information from a longitudinal resting-state fMRI dataset from the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA), the GNN-TF enables a comprehensive analysis that captures critical aspects of longitudinal imaging data. Comparative analyses against a variety of established machine learning and deep learning models demonstrate that GNN-TF outperforms these state-of-the-art methods, delivering superior predictive accuracy for predicting future tobacco usage. The end-to-end, time-aware transformer fusion structure of the proposed GNN-TF model successfully integrates multiple data modalities and leverages temporal dynamics, making it a valuable analytic tool for functional brain imaging studies focused on clinical outcome prediction.