Few-Shot Deployment of Pretrained MRI Transformers in Brain Imaging Tasks

TL;DR

This paper introduces a practical framework for deploying pretrained MRI transformers in brain imaging tasks with minimal annotated data, achieving high accuracy and robustness across diverse applications.

Contribution

It presents a novel few-shot deployment method using MAE pretraining on large MRI datasets, enabling effective transfer to classification and segmentation tasks with limited supervision.

Findings

State-of-the-art accuracy in MRI sequence identification

Outperforms baselines in skull stripping and segmentation

Demonstrates efficiency and scalability in low-resource settings

Abstract

Machine learning using transformers has shown great potential in medical imaging, but its real-world applicability remains limited due to the scarcity of annotated data. In this study, we propose a practical framework for the few-shot deployment of pretrained MRI transformers in diverse brain imaging tasks. By utilizing the Masked Autoencoder (MAE) pretraining strategy on a large-scale, multi-cohort brain MRI dataset comprising over 31 million slices, we obtain highly transferable latent representations that generalize well across tasks and datasets. For high-level tasks such as classification, a frozen MAE encoder combined with a lightweight linear head achieves state-of-the-art accuracy in MRI sequence identification with minimal supervision. For low-level tasks such as segmentation, we propose MAE-FUnet, a hybrid architecture that fuses multiscale CNN features with pretrained MAE embeddings. This model consistently outperforms other strong baselines in both skull stripping and multi-class anatomical segmentation under data-limited conditions. With extensive quantitative and qualitative evaluations, our framework demonstrates efficiency, stability, and scalability, suggesting its suitability for low-resource clinical environments and broader neuroimaging applications.