Are Large Brainwave Foundation Models Capable Yet? Insights from Fine-tuning

TL;DR

This study evaluates large brainwave foundation models for brain-computer interface tasks, revealing limited improvements over traditional models and emphasizing the need for domain-specific development and efficient adaptation techniques.

Contribution

The paper systematically assesses LBMs in BCI tasks, introduces LoRA adaptation for these models, and highlights architectural inefficiencies limiting their current capabilities.

Findings

LBMs achieve marginal performance gains (0.9%-1.2%) over traditional models.

Parameter-efficient LoRA adaptation reduces trainable parameters without performance loss.

Architectural and training inefficiencies hinder LBMs' effectiveness in BCI applications.

Abstract

Foundation Models have demonstrated significant success across various domains in Artificial Intelligence (AI), yet their capabilities for brainwave modeling remain unclear. In this paper, we comprehensively evaluate current Large Brainwave Foundation Models (LBMs) through systematic fine-tuning experiments across multiple Brain-Computer Interface (BCI) benchmark tasks, including memory tasks and sleep stage classification. Our extensive analysis shows that state-of-the-art LBMs achieve only marginal improvements (0.9%-1.2%) over traditional deep architectures while requiring significantly more parameters (millions vs thousands), raising important questions about their efficiency and applicability in BCI contexts. Moreover, through detailed ablation studies and Low-Rank Adaptation (LoRA), we significantly reduce trainable parameters without performance degradation, while demonstrating that architectural and training inefficiencies limit LBMs' current capabilities. Our experiments span both full model fine-tuning and parameter-efficient adaptation techniques, providing insights into optimal training strategies for BCI applications. We pioneer the application of LoRA to LBMs, revealing that performance benefits generally emerge when adapting multiple neural network components simultaneously. These findings highlight the critical need for domain-specific development strategies to advance LBMs, suggesting that current architectures may require redesign to fully leverage the potential of foundation models in brainwave analysis.