Brain Foundation Models with Hypergraph Dynamic Adapter for Brain Disease Analysis

TL;DR

This paper introduces SAM-Brain3D, a comprehensive brain foundation model trained on extensive MRI data, combined with HyDA, a hypergraph-based adapter for efficient multi-modal and patient-specific brain disease analysis, outperforming existing methods.

Contribution

The work presents a novel brain-specific foundation model and a hypergraph dynamic adapter for improved multi-modal, multi-scale, and personalized brain disease analysis.

Findings

Outperforms state-of-the-art methods in brain disease segmentation and classification.

Effectively fuses multi-modal data using hypergraphs for enhanced analysis.

Demonstrates robust generalization across diverse brain tasks.

Abstract

Brain diseases, such as Alzheimer's disease and brain tumors, present profound challenges due to their complexity and societal impact. Recent advancements in brain foundation models have shown significant promise in addressing a range of brain-related tasks. However, current brain foundation models are limited by task and data homogeneity, restricted generalization beyond segmentation or classification, and inefficient adaptation to diverse clinical tasks. In this work, we propose SAM-Brain3D, a brain-specific foundation model trained on over 66,000 brain image-label pairs across 14 MRI sub-modalities, and Hypergraph Dynamic Adapter (HyDA), a lightweight adapter for efficient and effective downstream adaptation. SAM-Brain3D captures detailed brain-specific anatomical and modality priors for segmenting diverse brain targets and broader downstream tasks. HyDA leverages hypergraphs to fuse complementary multi-modal data and dynamically generate patient-specific convolutional kernels for multi-scale feature fusion and personalized patient-wise adaptation. Together, our framework excels across a broad spectrum of brain disease segmentation and classification tasks. Extensive experiments demonstrate that our method consistently outperforms existing state-of-the-art approaches, offering a new paradigm for brain disease analysis through multi-modal, multi-scale, and dynamic foundation modeling.