TractGraphFormer: Anatomically Informed Hybrid Graph CNN-Transformer Network for Interpretable Sex and Age Prediction from Diffusion MRI Tractography

TL;DR

TractGraphFormer is a hybrid Graph CNN-Transformer model that effectively captures local and global white matter features from diffusion MRI tractography, improving sex and age prediction accuracy and interpretability.

Contribution

The paper introduces TractGraphFormer, a novel hybrid deep learning framework combining Graph CNN and Transformer modules for enhanced analysis of brain white matter in diffusion MRI.

Findings

Strong performance in large datasets of children and young adults.

Identification of consistent predictive white matter tracts for sex and age.

Integration of local anatomical and global features improves prediction accuracy.

Abstract

The relationship between brain connections and non-imaging phenotypes is increasingly studied using deep neural networks. However, the local and global properties of brain white matter networks are often overlooked in convolutional network design. We introduce TractGraphFormer, a hybrid Graph CNN-Transformer deep learning framework tailored for diffusion MRI tractography. This model leverages local anatomical characteristics and global feature dependencies of white matter structures. The Graph CNN module captures white matter geometry and grey matter connectivity to aggregate local features from anatomically similar white matter connections, while the Transformer module uses self-attention to enhance global information learning. Additionally, TractGraphFormer includes an attention module for interpreting predictive white matter connections. We apply TractGraphFormer to tasks of sex and age prediction. TractGraphFormer shows strong performance in large datasets of children (n=9345) and young adults (n=1065). Overall, our approach suggests that widespread connections in the WM are predictive of the sex and age of an individual. For each prediction task, consistent predictive anatomical tracts are identified across the two datasets. The proposed approach highlights the potential of integrating local anatomical information and global feature dependencies to improve prediction performance in machine learning with diffusion MRI tractography.