On the Role of Activation Functions in EEG-To-Text Decoder

TL;DR

This paper investigates how different activation functions affect EEG-to-text neural decoding performance, finding that polynomial and leaky ReLU functions can improve results without changing the model architecture.

Contribution

It introduces a comparative analysis of activation functions in EEG-to-text decoding, highlighting the benefits of polynomial and leaky ReLU functions for model performance.

Findings

Polynomial activation improves 1-gram performance.

Learnable 3rd-degree activation outperforms non-learnable.

Leaky ReLU surpasses baseline on higher n-gram evaluations.

Abstract

In recent years, much interdisciplinary research has been conducted exploring potential use cases of neuroscience to advance the field of information retrieval. Initial research concentrated on the use of fMRI data, but fMRI was deemed to be not suitable for real-world applications, and soon, research shifted towards using EEG data. In this paper, we try to improve the original performance of a first attempt at generating text using EEG by focusing on the less explored area of optimising neural network performance. We test a set of different activation functions and compare their performance. Our results show that introducing a higher degree polynomial activation function can enhance model performance without changing the model architecture. We also show that the learnable 3rd-degree activation function performs better on the 1-gram evaluation compared to a 3rd-degree non-learnable function. However, when evaluating the model on 2-grams and above, the polynomial function lacks in performance, whilst the leaky ReLU activation function outperforms the baseline.