Real-time brain machine interaction via social robot gesture control

TL;DR

This paper presents a real-time brain-machine interface that enables users to control a social robot through EEG-based neural decoding, facilitating neurorehabilitation and mental training with immediate feedback.

Contribution

It introduces a novel neurofeedback-based BCI platform that integrates social robot gesture control for cognitive rehabilitation and demonstrates real-time interaction capabilities.

Findings

Successful decoding of imagined movements to control a cursor.

Real-time interaction with a social robot via EEG signals.

Potential for enhancing mental abilities through neurofeedback.

Abstract

Brain-Machine Interaction (BMI) system motivates interesting and promising results in forward/feedback control consistent with human intention. It holds great promise for advancements in patient care and applications to neurorehabilitation. Here, we propose a novel neurofeedback-based BCI robotic platform using a personalized social robot in order to assist patients having cognitive deficits through bilateral rehabilitation and mental training. For initial testing of the platform, electroencephalography (EEG) brainwaves of a human user were collected in real time during tasks of imaginary movements. First, the brainwaves associated with imagined body kinematics parameters were decoded to control a cursor on a computer screen in training protocol. Then, the experienced subject was able to interact with a social robot via our real-time BMI robotic platform. Corresponding to subject's imagery performance, he/she received specific gesture movements and eye color changes as neural-based feedback from the robot. This hands-free neurofeedback interaction not only can be used for mind control of a social robot's movements, but also sets the stage for application to enhancing and recovering mental abilities such as attention via training in humans by providing real-time neurofeedback from a social robot.