Acting Thoughts: Towards a Mobile Robotic Service Assistant for Users with Limited Communication Skills

TL;DR

This paper introduces a novel brain-computer interface framework enabling paralyzed users to control a robotic assistant through thoughts, integrating signal decoding, planning, and perception for real-world fetch-and-carry tasks.

Contribution

It presents a comprehensive, real-time BCI system for robotic control that is robust and adaptable to dynamic environments, expanding human-robot interaction possibilities.

Findings

System successfully performs fetch-and-carry tasks.

Robustness demonstrated in real-world scenarios.

Adapts to environmental changes effectively.

Abstract

As autonomous service robots become more affordable and thus available also for the general public, there is a growing need for user friendly interfaces to control the robotic system. Currently available control modalities typically expect users to be able to express their desire through either touch, speech or gesture commands. While this requirement is fulfilled for the majority of users, paralyzed users may not be able to use such systems. In this paper, we present a novel framework, that allows these users to interact with a robotic service assistant in a closed-loop fashion, using only thoughts. The brain-computer interface (BCI) system is composed of several interacting components, i.e., non-invasive neuronal signal recording and decoding, high-level task planning, motion and manipulation planning as well as environment perception. In various experiments, we demonstrate its applicability and robustness in real world scenarios, considering fetch-and-carry tasks and tasks involving human-robot interaction. As our results demonstrate, our system is capable of adapting to frequent changes in the environment and reliably completing given tasks within a reasonable amount of time. Combined with high-level planning and autonomous robotic systems, interesting new perspectives open up for non-invasive BCI-based human-robot interactions.