# A Modality-agnostic Multi-task Foundation Model for Human Brain Imaging

## TL;DR

BrainFM is a versatile, modality-agnostic foundation model for human brain imaging that effectively handles diverse imaging modalities and tasks, demonstrating robustness and broad applicability in clinical settings.

## Contribution

The paper introduces BrainFM, a novel multi-task foundation model that generalizes across multiple brain imaging modalities and tasks using a new training strategy, addressing previous modality sensitivity issues.

## Key findings

- Effective across five brain imaging tasks.
- Robust performance on eleven public datasets.
- Handles diverse modalities and image artifacts.

## Abstract

Recent learning-based approaches have made astonishing advances in calibrated medical imaging like computerized tomography (CT), yet they struggle to generalize in uncalibrated modalities -- notably magnetic resonance (MR) imaging, where performance is highly sensitive to the differences in MR contrast, resolution, and orientation. This prevents broad applicability to diverse real-world clinical protocols. Here we introduce BrainFM, a modality-agnostic, multi-task vision foundation model for human brain imaging. With the proposed "mild-to-severe" intra-subject generation and "real-synth" mix-up training strategy, BrainFM is resilient to the appearance of acquired images (e.g., modality, contrast, deformation, resolution, artifacts), and can be directly applied to five fundamental brain imaging tasks, including image synthesis for CT and T1w/T2w/FLAIR MRI, anatomy segmentation, scalp-to-cortical distance, bias field estimation, and registration. We evaluate the efficacy of BrainFM on eleven public datasets, and demonstrate its robustness and effectiveness across all tasks and input modalities. Code is available at https://github.com/jhuldr/BrainFM.