Entropy-based Motion Intention Identification for Brain-Computer Interface

TL;DR

This paper introduces an entropy-based method for real-time detection of motion intention in Brain-Computer Interfaces, achieving high accuracy and early prediction of user intent from EEG signals.

Contribution

It presents a novel entropy-based approach for continuous motion intention identification in BCIs, enabling early and reliable detection of user commands.

Findings

Achieves 80% accuracy in real-time motion prediction at 8 Hz

Detects motion intention over 1 second before muscular activation

Effective in identifying Intentional Non-Control states

Abstract

The identification of intentionally delivered commands is a challenge in Brain Computer Interfaces (BCIs) based on Sensory-Motor Rhythms (SMR). It is of fundamental importance that BCI systems controlling a robotic device (i.e., upper limb prosthesis) are capable of detecting if the user is in the so called Intentional Non-Control (INC) state (i.e., holding the prosthesis in a given position). In this work, we propose a novel approach based on the entropy of the Electroencephalogram (EEG) signals to provide a continuous identification of motion intention. Results from ten healthy subjects suggest that the proposed system can be used for reliably predicting motion in real-time at a framerate of 8 Hz with $80\% \pm 5\%$ of accuracy. Moreover, motion intention can be detected more than 1 second before muscular activation with an average accuracy of $76\% \pm 11\%$.

Equations

$$H_{sh}=-\sum _i{P}_i\log P_i$$

$$n{H}_{sh}=\frac{H_{sh}}{\log k}$$

$$p_t(y_k|x)=\frac{p_t(x|y_k)p(y_k)}{{\sum }_j{p}_t(x|y_j)p(y_j)}$$

$$D(y_t)=\alpha \cdot D(y_{t-1})+(1-\alpha )\cdot p(y_t|x_t)$$

Full text

I INTRODUCTION

In the last years, thanks to a deeper understanding of the mechanism regulating our brain functions, several Brain-Computer Interface (BCI) technologies have been developed to allow people with severe disabilities to interact with devices for control or for communication [1]. Among those, asynchronous BCIs based on Sensory-Motor Rhythms (SMR) have become more sophisticated and reliable both in terms of classification accuracy and speed. Such performances enabled the proliferation of Motor Imagery (MI) BCIs, allowing a more natural control of robotic devices. Recently, researchers started exploiting the principle of ”sharing autonomy” [2] in controlling robotic devices, both for mobility [3] and manipulation [4]. Shared-control approaches allow the user to focus the attention on delivering a reduced number of high-level goals, leaving low level problems to the robotic intelligence (i.e. obstacle avoidance). These solutions have shown to reduce the number of commands needed to navigate a mobile robot through complex environments [5].

High classification accuracy of SMR-BCIs is achieved when the user is intentionally delivering a command (e.g., by discriminating if the user is imagining movements of the hands or the feet). It is also important to design BCI systems able to recognize when the user does not want to deliver any command, in order not to change the behavior of the robot (i.e., keep an upper limb prosthesis in a given position). This particular situation is known as Intentional Non-Control (INC) state [6, 7]. In the past, few solutions have been proposed to face the problem of detecting INC state, and they can be principally divided in three groups: i) leaving to the user the burden of actively controlling the BCI to not deliver any command; ii) exploiting multi-class classification techniques, introducing an extra class that represent the resting state [8, 9]; iii) adopting control strategies that minimize the delivery of undesired commands [6, 10]. However all these solutions present some limitations: the first implicates a higher workload for the user to be in control of the robotic device; the second could compromise the classification performance of conventional SMR-BCIs due to the difficulty of consistently modelling the unbounded rest class; the third is a promising approach but few people are considering it at present time.

In this paper, we propose a novel approach for motion intention detection based on the entropy of Electroencephalogram (EEG) signals in order to continuously identify periods of Intentional Control (IC) and Intentional Non-Control (INC). The entropy computed from EEG signals have been previously used in several studies to discriminate among different (active) motor imagery tasks [11, 12, 13]. However, none of these works faces the challenge of INC detection. They all exploit the entropy-based features for conventional MI BCI (i.e., imagining right vs. left hands after a cue) as an alternative to the most common frequency-based features. Herein, we hypothesize the complexity (i.e., entropy) of EEG signals might convey orthogonal information about the intention to move. In a previous work [7], we have shown that an entropy-based INC detector coupled to a SMR-BCI could reduce the amount of unintentionally delivered commands of about $43\%$ on average, keeping unaltered the performance of the MI task. However, in that study, as well as in most of the MI studies, the experimental protocol consists of cue-guided discrete trials, where the subject starts at rest and after a certain cue appears has to suddenly start performing the task. In this work we aim to extend the previously proposed system to detect self-paced movements identifying in real-time continuous switching between states of IC and INC while controlling an external device. The performances of the proposed detector are evaluated in classifying periods of movements execution as indicated by Electromyography (EMG) signals, but also in predicting motion intention before the generation of muscular activity.

This paper is organized as follows. Section II describes the experimental protocol and the BCI system. Section III presents the obtained results, discussed in details in Section IV. Finally Section V provides some conclusions and our future works.

II METHODS

II-A Experimental design

Ten healthy volunteers participated to this study (age from 18 to 58 years old, 4 female). Participants had no known significant health problems at the time of the experiment. All subjects were right-handed and only two subjects (s02 and s07) were familiar with BCI. The study was carried out in accordance with the principles of the Declaration of Helsinki.

The experimental protocol consisted in a videogame in which subjects were asked to ”shoot” to a white rectangle (the target) moving from right to left at a constant speed. Users can ”shoot” to the target by means of a stylized gun placed in the middle of the screen. The shooting is controlled through muscular activity (EMG) of the right and left forearm, by slightly closing the fingers of both hands simultaneously. At the same time, EEG signals were acquired and synchronized with the game interface through a custom event sender from Robot Operating System (ROS) to the BCI library. A screenshot of the experimental protocol is shown in Figure 1A. The game is structured in 4 consecutive phases, shown in Figure 1B and following explained:

• •

INC-stationary: no target is present on the screen or the target has just left the middle of the screen. Subjects are asked to not perform any movement;

• •

IC-cue: the left edge of the target appears on the screen, but it has not reached the middle of the screen yet. Subjects are asked not to perform any movement, but they know they should start moving in few seconds;

• •

IC-stationary: the target reaches the middle of the screen and subjects are asked to shoot to the rectangle, continuously opening and closing both hands;

• •

INC-cue: the right edge of the target appears on the screen, but it has not reached the middle of the screen yet. Subjects are asked to continue shooting to the target, but they know the should stop moving in few seconds.

Visual feedbacks are provided to inform subjects if they are behaving correctly, as shown in Figure 1D: the target turns to green if hit when it is required (IC-stationary, INC-cue) or to red if missed; small red rectangles appear if subjects move while not required (INC-stationary, IC-cue). Each cue condition (IC-cue, INC-cue) lasted $3.0$ seconds, while stationary conditions (IC-stationary, INC-stationary) lasted a random time between $3.5$ and $4.5$ seconds.

Each run of the game consists of the $10$ trials of each condition. Before starting the experiment, each subject tried one run to familiarize with the game. Then, five runs were performed for each subject, resulting in a total of $200$ trials per subject.

II-B EMG recording

The EMG signals were acquired through two Myo Gesture Control Armbands, by Thalmic Labs. Both Myo sensors have been connected and synchronized to ROS by means of an open source custom software library111The library code is publicly available at https://github.com/ste92uo/ROS_multipleMyo. The envelop of EMG signals has been extracted online by means of rectification and smoothing to control the game interface and provide closed-loop feedbacks of subject’s movements. In order to shoot, the EMG envelops of both forearms should overcome a threshold, manually tuned for each subject in order to have a highly sensitive interface. For classifier training, the EMG signals have been used to segment the acquired data in time periods when the subject is intentionally controlling the interface generating EMG activity and time periods when the subject is not contracting the muscles in order to voluntary not control the interface. Thus, data acquired in the cue phases have been merged with data from their corresponding stationary phases, as shown in Figure 1B–i.e. IC-cue with INC-stationary and INC-cue with IC-stationary.

II-C EEG recording and preprocessing

EEG data were recorded with an active 16-channels acquisition system (g.tec medical engineering, Austria) at $512$ Hz with reference on the right ear and ground on AFz electrode. Two electrodes configurations were considered: i) a ”Motor configuration” (Mc) in Figure 1C (left), that corresponds to the one used in our previous studies [5, 7], covering the sensory-motor cortex (Fz, FC3, FC1, FCz, FC2, FC4, C3, C1, Cz, C2, C4, CP3, CP1, CPz, CP2, CP4); ii) a ”Fronto-Parietal configuration” (FPc) in Figure 1C (right), with electrodes more spread over the cortex, covering also frontal and parietal regions (F3, F1, Fz, F2, F4, FCz, C3, C1, Cz, C2, C4, P3, P1, Pz, P2, P4). At admission, the participants were randomly assigned to one of these two configurations. Five subjects performed the experiment with one configuration and the remaining five with the other configuration. EEG signals from each channel were filtered with a digital notch filter at $50$ Hz by the amplifier, to remove the power line interference, and a Common Average Reference (CAR) spatial filter was applied.

II-D Entropy extraction

The entropy analysis has been performed offline on Matlab, simulating an online BCI loop with real-time constraints. The recorded data were segmented with a sliding window of $1.5$ seconds and an overlap of $0.125$ seconds, resulting in a framerate of $8$ Hz. Six different frequency bands have been extracted from the filtered EEG signals (8-13, 14-22, 22-30, 30-45, 2-45 and 8-30 Hz) with a fourth-order zero-lag Butterworth band pass filter. Subsequently, the band-power in each frequency band has been calculated as the envelop of the Hilbert transform of the signal. The Shannon Entropy [14] has been computed for each frequency band. For continuous signals, the Shannon Entropy $H_{sh}$ is defined as

[TABLE]

where $i$ ranges over all amplitudes of the signal, $P_{i}$ indicates the probability of the signal having amplitude $a_{i}$. For discrete signals, the Shannon Entropy can be computed by means of histogram, linearly dividing the amplitude range of the signal into k bins. Finally, the normalized Shannon Entropy is used, computed as

[TABLE]

In this work, $k=32$ has been taken.

The most discriminative features, defined as channel-band pairs, to distinguish between IC and INC states have been found for each subject by means of Canonical Variate Analysis (CVA) [15]. In this study, CVA has been used to extract the canonical discriminant spatial patterns (CDSP), representing the features whose directions maximize the difference in mean Shannon Entropy between the classes. The channel-band pairs that contributed the most on the CDSP are identified through a Discrimimation index, computed from the pooled correlation matrix between each feature and the CDSP.

A statistical Gaussian classifier was used to compute the posterior class probabilities of each frame; i.e. the probability that at the current frame, the computed entropy over the $1.5$ seconds time window belongs to each class (IC or INC). Each class is represented by $N_{p}$ Gaussian prototypes with equal weight $1/N_{p}$, thus the class-conditional probability density function $p_{t}(x|y_{k})$ for the class $y_{k}$ at time $t$ can be computed as the superimposition of the Gaussian prototypes. The posterior probability of each class is

[TABLE]

All classes are assumed to have equal prior probabilities $p(y_{k})$. The Gaussian prototypes were initialized by clustering algorithm, namely Self-Organizing Maps (SOM) [16]. Then, the initial estimation is iteratively improved by means of stochastic gradient descent minimizing the mean square error $E=(1/2)\sum_{i}{(y_{i}-t_{i})^{2}}$, where $y_{i}$ is the predicted class and $t_{i}$ is the target class. More details can be found in [17].

In order to smooth the outcomes of the classifier, an exponential integrator has been used to accumulate the evidence over time with the following decision making formula

[TABLE]

where $D(y_{t})$ is the previous level of decision making and $\alpha$ is the integration parameter. Finally, the decision on the predicted class is made by a thresholding procedure with hysteresis. In this sense, if the actual decision level $D(y_{t})$ overcomes a threshold $th$, the subject is considered in the state of IC; if the decision level goes below $1-th$, the subject is considered in the state of INC; otherwise, if the decision level is between $th$ and $1-th$, the previous decision is kept.

III RESULTS

III-A Motion execution detection (IC vs. INC)

The first three runs of each subject has been used as training dataset for features selection and classifier training. Performances have been evaluated on the last two sessions. Table I shows the performance for each subject of the proposed entropy-based BCI in detecting periods of movement execution (IC) from periods of resting (INC). For comparison, the same dataset has been also classified by a conventional SMR-BCI based on the PSD of the EEG signals [17]. From the results, our system achieved comparable performance and better classification accuracy for 7 out of 10 subjects, with an average improve of $3.3\%\pm 5.5\%$, even if not statistically significant (t-test, p$>$0.05). The group of subjects belonging to the ’FPc’ configuration shows on average slightly better performances compared to subjects wearing the ’Mc’ configuration, with classification accuracy of $82.0\%\pm 4.6\%$ and $77.0\%\pm 4.4\%$ respectively for the entropy-based BCI, and $79.0\%\pm 3.8\%$ and $73.0\%\pm 9.5\%$ respectively for the PSD-based BCI, but with no statistically significant difference (t-test, p$>$0.05). It is worth mentioning, that the results refer to the single sample accuracy of the testing sessions, thus providing a classification output at a rate of 8 Hz, as shown in Figure 2. The red line represents the posterior probabilities of the IC state after the exponential integration. When the integrated probabilities crosses the thresholds (black dotted lines), the corresponding prediction is generated (green line). The integration parameter $\alpha$ and the decision threshold $th$ were identified for each subject by means of discrete grid search and the best pair of parameters ($\hat{\alpha}$, $\hat{th}$) are selected as the one maximizing the prediction accuracy in a 3-fold cross-validation over the training dataset. $\hat{\alpha}$ equal to 0.9 and $\hat{th}$ equal to 0.65 has been selected for almost all the subjects (8 out of 10 subjects). The results show the feasibility of using the proposed entropy-based BCI for real-time detection of intentional movements execution locked to the EMG activity (blue line) with maximum accuracy of $85\%$ for s08 and s09 and minimum accuracy of $73\%$ for s05.

III-B Topographic distributions

As expected from the results in [7], the intentional execution of a movement (IC state) generates an increase of the complexity in the brain activity, as measured by the entropy of the EEG signals, compared to resting periods (INC state), as shown in Figure 3A. Figure 3B shows the topographic maps for each subject in the frequency band that has been selected the most by CVA. Red areas represent locations where the difference between the entropy in the IC and INC states is higher, and blue areas where no significant difference is evident. Black circled dots represent the channels selected by CVA in the shown frequency band. Results show that intentional movements execution is discriminated mostly by frontal and parietal electrodes for both configurations, ’FPc’ (top) and ’Mc’ (bottom), except for C4 that has been selected in four out of ten subjects. This tendency has been confirmed by analyzing the percentage of times the electrodes in each line have been selected by CVA for all the subjects (Figure 4). It can be seen that for the entropy-based BCI, CVA selected more than $50\%$ of the features in channels from lines F and P, about $35\%$ in channels from lines FC and CP, and only $14\%$ from the central line (C), even if it was present in both configurations. On the other hand, by applying CVA on PSD features, more than $50\%$ of the features are selected from channels in the central (C) and centro-parietal (CP) lines, while less than $25\%$ in the frontal (F) and parietal (P) electrodes.

III-C Motion preparation detection (INC vs. IC-cue)

As shown in Figure 3A, the difference between the entropy in the IC and INC states is evident already during the appearance of their cue phases (IC-cue and INC-cue respectively), and then reaching a plateau from the beginning of their stationary phases. Indeed, from the analysis of the spectrograms of channels related to movements of both hands (C3, C1, C2 and C4) averaged across all trials of every subject, it can be seen an Event-Related Desynchronization (ERD) after the beginning of the IC-cue phase, particularly evident in $\beta$ band (14-22 Hz). Paired t-tests have confirmed a statistically significant difference (p$<$0.0001) between the spectrogram 3 seconds before the IC-cue (INC phase) and in the 3 seconds of the IC-cue phase, for all the analyzed electrodes in both $\mu$ and $\beta$ bands. These evidences suggest that motion intention of self-paced movements can be detected from the brain activity before muscular activation. To this aim, the proposed entropy-based BCI has been trained to classify between INC state and IC-cue state. The results are reported for each subject in Table II for the single sample classification and the single trial classification. The single sample classification accuracy show that the entropy of the EEG signals can be used to identify in real-time with a framerate of 8 Hz motion intention before muscles contraction with performance slightly lower respect to the detection of movement execution. The system provides an overall classification accuracy of $74\%\pm 5\%$ with maximum performance of $82\%$ for s04. The single trial classification performance are measured as the accuracy and the delay in predicting motion intention after the appearance of the IC-cue. Thus, as shown in Figure 6, if the system identifies motion intention in the 3 seconds of the IC-cue phase, the trial is considered correctly predicted; if motion intention is predicted in the 3 seconds before the IC-cue phase (INC phase), the trial is considered incorrect; otherwise, if no motion intention is predicted before the end of the IC-cue phase, the trial is considered as missed. The proposed BCI is able to predict the intention of performing a motion with a single trial accuracy of $76\%\pm 11\%$ in $1.7\pm 0.7$ seconds on average after the beginning of the IC-cue phase. On average, $15\%\pm 6\%$ of the trials are missed, while less than $9\%$ are classified before the cue, showing the robustness of the system to missclassifications.

IV DISCUSSION

As mentioned in the introduction, providing a continuous and robust identification of motion intention is a challenge in BCI systems. Ideally, the INC detector should be able to accurately identify when the user wants to intentionally control the BCI independently from the specific motor tasks to be performed. Thus, the system should recognize the intention of performing a general motion. The results in this paper strengthen the hypothesis presented in [7] of the existence of a correlation between the complexity of the EEG signals, measured by means of Shannon Entropy, and the fact that the user is resting (INC state) or performing a movement-related task (IC state). The Shannon Entropy has been previously used to measure the user’s level of attention during SMR tasks [18]. In particular, high entropy values are related to higher attention and fast delivery of BCI commands, while low entropy values to slow delivery of BCI commands due to reduced attention. From neuroscientific literature [19], we would expect movement-related features in $\mu$ and $\beta$ bands to be more evident right above the motor cortex. This fact has been confirmed also in our study by looking to the channels selected from the PSD features, shown in Figure 4 (right). Interestingly, the difference in brain complexity between INC and IC states is more evident in electrodes from frontal and parietal areas in both configurations as shown in Figure 4 (left). On the other hand, only few information is carried out by central electrodes, suggesting that the entropy provides different additional information to motion detection respect to the spectrogram analysis, that could improve the identification of self-paced motion intention, as shown in Table I. Other studies [20] have confirmed the role of frontal and parietal areas in the generation of ”intentionality” in executing movements. They have shown that the network related to motion intention generation starts from the pre-supplementary motor area (pre-SMA) in the frontal cortex, passes through the precuneous in the parietal cortex up to the pre-motor and motor cortex for movement execution [21]. In this sense, our results suggest that the entropy of the EEG signals in frontal and parietal regions could be used as signature of motion intention, independently from the specific motion task.

The first objective of this study was to predict motion execution locked to EMG activity. According to the results in Figure 2, the integrated probabilities computed from the entropy-based features robustly follow the dynamic of real movements, continuously identify when the user starts and ends the execution of a movement with time delays in the order of few seconds ($2.5\pm 3.4$ sec). Thus, the entropy-based BCI could be used as an alternative to or in conjunction with EMG-triggered neuroprostheses for people with severe motor disabilities and muscles weakness [22].

Previous works highlighted that $\beta$ ERD could be associated with motor planning and other functions related to an attentive state of the brain during motor imagery tasks [23]. The statistical tests on the spectrogram of the signal before and after the appearance of the IC-cue prove the existence of a self-paced motion preparation prior to movement onset. Results in Table II shows acceptable classification accuracy of movement preparation respect to resting phases (INC vs. IC-cue), with comparable accuracy to the detection of movement execution (IC vs. INC). It is worth highlighting that, differently from the classification of movement executions that were guided by the dynamics of the game interface (i.e., the subject should start and stop shooting immediately after the left and right edges of the target cross the middle of the screen, respectively), movement preparation was self-paced since the subject could start planning the movement at anytime after the appearance of the IC-cue, making its identification more challenging. These results suggest the feasibility of exploiting the proposed entropy-based BCI for self-paced motor imagery BCI applications, since it is able to predict motion intention also when no muscle activity is generated.

V CONCLUSIONS

This study aims to provide a novel BCI system for continuous prediction of Intentional Control (IC) and Intentional Non-Control (INC) states. The results reveal the possibility to improve the prediction of self-paced motion intention from the complexity of brain activity, measured as the entropy of EEG signals, principally in $\alpha$ and $\beta$ bands. The proposed entropy-based BCI provides comparable performance in detecting motion execution (IC vs. INC) and motion preparation (INC vs. IC-cue), making it feasible to be used for the detection of active motor imagery tasks. Indeed, single trial classification performance shows that motion intention could be reliably predicted more than 1 second before the generation of muscular activity. Moreover, topographic maps in this study and evidences from literature suggest that the analysis of the connectivity between frontal and parietal areas could deep the understanding of the generation of motion intention, independently from the specific motor task to be executed. In future works, we are planning to evaluate the performance of the proposed method as INC detector in a two-class SMR-BCI, in order to reduce the amount of unintentionally delivered commands and improve the control of robotic devices with shared autonomy.