Beware of Overestimated Decoding Performance Arising from Temporal Autocorrelations in Electroencephalogram Signals

TL;DR

This paper reveals that high EEG decoding accuracies in BCI tasks may be artificially inflated due to intrinsic temporal autocorrelations, emphasizing the need for careful experimental design to ensure genuine performance evaluation.

Contribution

The study formulates a unified framework to identify how EEG temporal autocorrelations can lead to overestimated decoding performance, using watermelon EEG data to demonstrate this pitfall.

Findings

High decoding accuracy can be achieved with data split strategies that exploit autocorrelations.

Decoding success is driven by intrinsic EEG autocorrelation features, not neural responses.

Proper experimental design is crucial to avoid inflated BCI performance estimates.

Abstract

Researchers have reported high decoding accuracy (>95%) using non-invasive Electroencephalogram (EEG) signals for brain-computer interface (BCI) decoding tasks like image decoding, emotion recognition, auditory spatial attention detection, etc. Since these EEG data were usually collected with well-designed paradigms in labs, the reliability and robustness of the corresponding decoding methods were doubted by some researchers, and they argued that such decoding accuracy was overestimated due to the inherent temporal autocorrelation of EEG signals. However, the coupling between the stimulus-driven neural responses and the EEG temporal autocorrelations makes it difficult to confirm whether this overestimation exists in truth. Furthermore, the underlying pitfalls behind overestimated decoding accuracy have not been fully explained due to a lack of appropriate formulation. In this work, we formulate the pitfall in various EEG decoding tasks in a unified framework. EEG data were recorded from watermelons to remove stimulus-driven neural responses. Labels were assigned to continuous EEG according to the experimental design for EEG recording of several typical datasets, and then the decoding methods were conducted. The results showed the label can be successfully decoded as long as continuous EEG data with the same label were split into training and test sets. Further analysis indicated that high accuracy of various BCI decoding tasks could be achieved by associating labels with EEG intrinsic temporal autocorrelation features. These results underscore the importance of choosing the right experimental designs and data splits in BCI decoding tasks to prevent inflated accuracies due to EEG temporal autocorrelation.