Large Connectome Model: An fMRI Foundation Model of Brain Connectomes Empowered by Brain-Environment Interaction in Multitask Learning Landscape

TL;DR

This paper introduces a large-scale, multitask foundation model for fMRI brain connectomes that leverages brain-environment interactions and demographic data to improve clinical neuroimaging applications.

Contribution

The paper presents a novel multitask learning architecture for pretraining on extensive fMRI data combined with environmental variables, enhancing downstream neuroimaging tasks.

Findings

Effective in sex prediction and human behavior recognition.

Improves early diagnosis of neurological and psychiatric disorders.

Demonstrates promising results across multiple clinical applications.

Abstract

A reliable foundation model of functional neuroimages is critical to promote clinical applications where the performance of current AI models is significantly impeded by a limited sample size. To that end, tremendous efforts have been made to pretraining large models on extensive unlabeled fMRI data using scalable self-supervised learning. Since self-supervision is not necessarily aligned with the brain-to-outcome relationship, most foundation models are suboptimal to the downstream task, such as predicting disease outcomes. By capitalizing on rich environmental variables and demographic data along with an unprecedented amount of functional neuroimages, we form the brain modeling as a multitask learning and present a scalable model architecture for (i) multitask pretraining by tokenizing multiple brain-environment interactions (BEI) and (ii) semi-supervised finetuning by assigning pseudo-labels of pretrained BEI. We have evaluated our foundation model on a variety of applications, including sex prediction, human behavior recognition, and disease early diagnosis of Autism, Parkinson's disease, Alzheimer's disease, and {Schizophrenia}, where promising results indicate the great potential to facilitate current neuroimaging applications in clinical routines.