BrainMT: A Hybrid Mamba-Transformer Architecture for Modeling Long-Range Dependencies in Functional MRI Data

TL;DR

BrainMT is a hybrid deep learning framework combining Mamba blocks and transformers to effectively model long-range spatiotemporal dependencies in fMRI data, leading to superior phenotypic prediction performance.

Contribution

This paper introduces BrainMT, a novel hybrid architecture that efficiently captures long-range dependencies in fMRI data using a two-stage process with Mamba and transformer blocks.

Findings

Achieves state-of-the-art results on sex and intelligence prediction tasks.

Outperforms existing methods significantly on large-scale datasets.

Demonstrates effective modeling of complex fMRI relationships.

Abstract

Recent advances in deep learning have made it possible to predict phenotypic measures directly from functional magnetic resonance imaging (fMRI) brain volumes, sparking significant interest in the neuroimaging community. However, existing approaches, primarily based on convolutional neural networks or transformer architectures, often struggle to model the complex relationships inherent in fMRI data, limited by their inability to capture long-range spatial and temporal dependencies. To overcome these shortcomings, we introduce BrainMT, a novel hybrid framework designed to efficiently learn and integrate long-range spatiotemporal attributes in fMRI data. Our framework operates in two stages: (1) a bidirectional Mamba block with a temporal-first scanning mechanism to capture global temporal interactions in a computationally efficient manner; and (2) a transformer block leveraging self-attention to model global spatial relationships across the deep features processed by the Mamba block. Extensive experiments on two large-scale public datasets, UKBioBank and the Human Connectome Project, demonstrate that BrainMT achieves state-of-the-art performance on both classification (sex prediction) and regression (cognitive intelligence prediction) tasks, outperforming existing methods by a significant margin. Our code and implementation details will be made publicly available at this https://github.com/arunkumar-kannan/BrainMT-fMRI