Fusing Pretrained ViTs with TCNet for Enhanced EEG Regression

TL;DR

This paper introduces a novel method combining pretrained Vision Transformers with Temporal Convolutional Networks to significantly improve EEG regression accuracy and speed, setting new benchmarks in Brain-Computer Interface analysis.

Contribution

It presents a new integrated model that leverages ViTs and TCNet for enhanced EEG analysis, optimizing patch construction for better speed-accuracy tradeoffs.

Findings

RMSE reduced from 55.4 to 51.8 on EEGEyeNet dataset

Model achieves up to 4.32x faster processing speed

Sets new benchmark in EEG regression accuracy

Abstract

The task of Electroencephalogram (EEG) analysis is paramount to the development of Brain-Computer Interfaces (BCIs). However, to reach the goal of developing robust, useful BCIs depends heavily on the speed and the accuracy at which BCIs can understand neural dynamics. In response to that goal, this paper details the integration of pre-trained Vision Transformers (ViTs) with Temporal Convolutional Networks (TCNet) to enhance the precision of EEG regression. The core of this approach lies in harnessing the sequential data processing strengths of ViTs along with the superior feature extraction capabilities of TCNet, to significantly improve EEG analysis accuracy. In addition, we analyze the importance of how to construct optimal patches for the attention mechanism to analyze, balancing both speed and accuracy tradeoffs. Our results showcase a substantial improvement in regression accuracy, as evidenced by the reduction of Root Mean Square Error (RMSE) from 55.4 to 51.8 on EEGEyeNet's Absolute Position Task, outperforming existing state-of-the-art models. Without sacrificing performance, we increase the speed of this model by an order of magnitude (up to 4.32x faster). This breakthrough not only sets a new benchmark in EEG regression analysis but also opens new avenues for future research in the integration of transformer architectures with specialized feature extraction methods for diverse EEG datasets.