Predicting Human Brain States with Transformer

TL;DR

This paper investigates using transformer models to predict human brain states from fMRI data, demonstrating accurate short-term predictions and reflecting brain connectome architecture, thus opening avenues for generative brain models.

Contribution

It introduces a transformer-based approach for predicting fMRI-derived brain states, a novel application of self-attention in neuroimaging analysis.

Findings

Accurately predicts brain states up to 5.04 seconds ahead

Predictions reflect the architecture of the brain's functional connectome

Demonstrates potential for generative models of brain activity

Abstract

The human brain is a complex and highly dynamic system, and our current knowledge of its functional mechanism is still very limited. Fortunately, with functional magnetic resonance imaging (fMRI), we can observe blood oxygen level-dependent (BOLD) changes, reflecting neural activity, to infer brain states and dynamics. In this paper, we ask the question of whether the brain states rep-resented by the regional brain fMRI can be predicted. Due to the success of self-attention and the transformer architecture in sequential auto-regression problems (e.g., language modelling or music generation), we explore the possi-bility of the use of transformers to predict human brain resting states based on the large-scale high-quality fMRI data from the human connectome project (HCP). Current results have shown that our model can accurately predict the brain states up to 5.04s with the previous 21.6s. Furthermore, even though the prediction error accumulates for the prediction of a longer time period, the gen-erated fMRI brain states reflect the architecture of functional connectome. These promising initial results demonstrate the possibility of developing gen-erative models for fMRI data using self-attention that learns the functional or-ganization of the human brain. Our code is available at: https://github.com/syf0122/brain_state_pred.