AdaBrain-Bench: Benchmarking Brain Foundation Models for Brain-Computer Interface Applications

TL;DR

AdaBrain-Bench is a comprehensive benchmark designed to evaluate the performance and generalizability of brain foundation models across diverse non-invasive BCI tasks, facilitating progress in neural decoding research.

Contribution

The paper introduces AdaBrain-Bench, a large-scale, standardized benchmark with datasets, evaluation metrics, and tools for assessing brain foundation models in BCI applications.

Findings

Evaluated multiple brain foundation models across diverse BCI tasks.

Provided insights into model selection for different transfer scenarios.

Established a reproducible, extensible benchmarking platform.

Abstract

Non-invasive Brain-Computer Interfaces (BCI) offer a safe and accessible means of connecting the human brain to external devices, with broad applications in home and clinical settings to enhance human capabilities. However, the high noise level and limited task-specific data in non-invasive signals constrain decoding capabilities. Recently, the adoption of self-supervised pre-training is transforming the landscape of non-invasive BCI research, enabling the development of brain foundation models to capture generic neural representations from large-scale unlabeled electroencephalography (EEG) signals with substantial noises. However, despite these advances, the field currently lacks comprehensive, practical and extensible benchmarks to assess the utility of the public foundation models across diverse BCI tasks, hindering their widespread adoption. To address this challenge, we present AdaBrain-Bench, a large-scale standardized benchmark to systematically evaluate brain foundation models in widespread non-invasive BCI tasks. AdaBrain-Bench encompasses a diverse collection of representative BCI decoding datasets spanning 7 key applications. It introduces a streamlined task adaptation pipeline integrated with multi-dimensional evaluation metrics and a set of adaptation tools. The benchmark delivers an inclusive framework for assessing generalizability of brain foundation models across key transfer settings, including cross-subject, multi-subject, and few-shot scenarios. We leverage AdaBrain-Bench to evaluate a suite of publicly available brain foundation models and offer insights into practices for selecting appropriate models in various scenarios. We make our benchmark pipeline available to enable reproducible research and external use, offering a continuously evolving platform to foster progress toward robust and generalized neural decoding solutions.