STAMP: Spatial-Temporal Adapter with Multi-Head Pooling

TL;DR

This paper introduces STAMP, a lightweight spatial-temporal adapter that enhances general time series models for EEG data, achieving performance comparable to specialized EEG models across multiple clinical datasets.

Contribution

The paper presents STAMP, a novel adapter that enables general TSFMs to effectively model EEG data, bridging the gap between general and EEG-specific foundation models.

Findings

STAMP achieves performance comparable to state-of-the-art EEGFMs.

The adapter is lightweight and flexible, supporting various inputs.

Comprehensive analysis on 8 EEG datasets demonstrates effectiveness.

Abstract

Time series foundation models (TSFMs) pretrained on data from multiple domains have shown strong performance on diverse modeling tasks. Various efforts have been made to develop foundation models specific to electroencephalography (EEG) data, which records brain electrical activity as time series. However, no comparative analysis of EEG-specific foundation models (EEGFMs) versus general TSFMs has been performed on EEG-specific tasks. We introduce a novel Spatial-Temporal Adapter with Multi-Head Pooling (STAMP), which leverages univariate embeddings produced by a general TSFM, implicitly models spatial-temporal characteristics of EEG data, and achieves performance comparable to state-of-the-art EEGFMs. A comprehensive analysis is performed on 8 benchmark datasets of clinical tasks using EEG for classification, along with ablation studies. Our proposed adapter is lightweight in trainable parameters and flexible in the inputs it can accommodate, supporting easy modeling of EEG data using TSFMs.