EEG-Based Analysis of Brain Responses in Multi-Modal Human-Robot Interaction: Modulating Engagement

TL;DR

This study demonstrates that a multi-modal human-robot interaction protocol, combining visual, motor, cognitive, and auditory tasks, significantly enhances user engagement as measured by EEG biomarkers, compared to traditional motor-only approaches.

Contribution

Introduces a novel multi-modal robotic interaction protocol that integrates multiple sensory and cognitive tasks to improve user engagement in human-robot interaction.

Findings

EEG biomarkers, especially relative alpha power, improved significantly during multi-modal tasks.

Engagement remained stable over time in the multi-modal protocol, unlike in motor-only tasks.

First neural evidence showing benefits of combined sensory-cognitive-motor interventions in healthy subjects.

Abstract

User engagement, cognitive participation, and motivation during task execution in physical human-robot interaction are crucial for motor learning. These factors are especially important in contexts like robotic rehabilitation, where neuroplasticity is targeted. However, traditional robotic rehabilitation systems often face challenges in maintaining user engagement, leading to unpredictable therapeutic outcomes. To address this issue, various techniques, such as assist-as-needed controllers, have been developed to prevent user slacking and encourage active participation. In this paper, we introduce a new direction through a novel multi-modal robotic interaction designed to enhance user engagement by synergistically integrating visual, motor, cognitive, and auditory (speech recognition) tasks into a single, comprehensive activity. To assess engagement quantitatively, we compared multiple electroencephalography (EEG) biomarkers between this multi-modal protocol and a traditional motor-only protocol. Fifteen healthy adult participants completed 100 trials of each task type. Our findings revealed that EEG biomarkers, particularly relative alpha power, showed statistically significant improvements in engagement during the multi-modal task compared to the motor-only task. Moreover, while engagement decreased over time in the motor-only task, the multi-modal protocol maintained consistent engagement, suggesting that users could remain engaged for longer therapy sessions. Our observations on neural responses during interaction indicate that the proposed multi-modal approach can effectively enhance user engagement, which is critical for improving outcomes. This is the first time that objective neural response highlights the benefit of a comprehensive robotic intervention combining motor, cognitive, and auditory functions in healthy subjects.