Attention-Based Variational Framework for Joint and Individual Components Learning with Applications in Brain Network Analysis

TL;DR

This paper introduces CM-JIVNet, a probabilistic model that uses attention mechanisms to effectively integrate and disentangle shared and modality-specific features from brain connectome data, improving analysis of brain networks.

Contribution

The paper presents a novel attention-based variational framework for joint and individual component learning in multimodal brain data integration, addressing high dimensionality and non-linearity challenges.

Findings

Outperforms existing methods in cross-modal reconstruction.

Accurately predicts behavioral traits from brain data.

Effectively disentangles shared and modality-specific features.

Abstract

Brain organization is increasingly characterized through multiple imaging modalities, most notably structural connectivity (SC) and functional connectivity (FC). Integrating these inherently distinct yet complementary data sources is essential for uncovering the cross-modal patterns that drive behavioral phenotypes. However, effective integration is hindered by the high dimensionality and non-linearity of connectome data, complex non-linear SC-FC coupling, and the challenge of disentangling shared information from modality-specific variations. To address these issues, we propose the Cross-Modal Joint-Individual Variational Network (CM-JIVNet), a unified probabilistic framework designed to learn factorized latent representations from paired SC-FC datasets. Our model utilizes a multi-head attention fusion module to capture non-linear cross-modal dependencies while isolating independent, modality-specific signals. Validated on Human Connectome Project Young Adult (HCP-YA) data, CM-JIVNet demonstrates superior performance in cross-modal reconstruction and behavioral trait prediction. By effectively disentangling joint and individual feature spaces, CM-JIVNet provides a robust, interpretable, and scalable solution for large-scale multimodal brain analysis.