NeuroAiR: Deep Learning Framework for Airwriting Recognition from Scalp-recorded Neural Signals

TL;DR

This paper introduces NeuroAiR, a deep learning framework that uses EEG signals to recognize airwritten English alphabets, demonstrating potential for non-invasive, gesture-free human-computer interaction with promising accuracy levels.

Contribution

The study constructs the first EEG-based airwriting dataset and explores multiple features and models, establishing a baseline for future EEG airwriting recognition research.

Findings

Highest accuracy of 44.04% with ICA features and EEGNet model

Various EEG features and frequency bands significantly impact recognition performance

Demonstrates feasibility of EEG-based airwriting as an alternative input modality

Abstract

Airwriting recognition is a task that involves identifying letters written in free space using finger movement. It is a special case of gesture recognition, where gestures correspond to letters in a specific language. Electroencephalography (EEG) is a non-invasive technique for recording brain activity and has been widely used in brain-computer interface applications. Leveraging EEG signals for airwriting recognition offers a promising alternative input method for Human-Computer Interaction. One key advantage of airwriting recognition is that users don't need to learn new gestures. By concatenating recognized letters, a wide range of words can be formed, making it applicable to a broader population. However, there has been limited research in the recognition of airwriting using EEG signals, which forms the core focus of this study. The NeuroAiR dataset comprising EEG signals recorded during writing English uppercase alphabets is first constructed. Various features are then explored in conjunction with different deep learning models to achieve accurate airwriting recognition. These features include processed EEG data, Independent Component Analysis components, source-domain-based scout time series, and spherical and head harmonic decomposition-based features. Furthermore, the impact of different EEG frequency bands on system performance is comprehensively investigated. The highest accuracy achieved in this study is 44.04% using Independent Component Analysis components and the EEGNet classification model. The results highlight the potential of EEG-based airwriting recognition as a user-friendly modality for alternative input methods in Human-Computer Interaction applications. This research sets a strong baseline for future advancements and demonstrates the viability and utility of EEG-based airwriting recognition.