Generalizability Under Sensor Failure: Tokenization + Transformers Enable More Robust Latent Spaces

TL;DR

This paper investigates the robustness of neural data representations to sensor failure in EEG data, demonstrating that tokenization combined with transformer models enhances generalizability and robustness compared to traditional CNNs.

Contribution

The study introduces a novel analysis of sensor failure robustness in EEG data using tokenization and transformers, supported by a new EEG dataset and comparative evaluation.

Findings

TOTEM outperforms or matches EEGNet across all generalizability scenarios.

Tokenization enables better generalization in neural data representations.

Analysis of TOTEM's latent codebook reveals the role of tokenization in robustness.

Abstract

A major goal in neuroscience is to discover neural data representations that generalize. This goal is challenged by variability along recording sessions (e.g. environment), subjects (e.g. varying neural structures), and sensors (e.g. sensor noise), among others. Recent work has begun to address generalization across sessions and subjects, but few study robustness to sensor failure which is highly prevalent in neuroscience experiments. In order to address these generalizability dimensions we first collect our own electroencephalography dataset with numerous sessions, subjects, and sensors, then study two time series models: EEGNet (Lawhern et al., 2018) and TOTEM (Talukder et al., 2024). EEGNet is a widely used convolutional neural network, while TOTEM is a discrete time series tokenizer and transformer model. We find that TOTEM outperforms or matches EEGNet across all generalizability cases. Finally through analysis of TOTEM's latent codebook we observe that tokenization enables generalization.