Autonomy Infused Teleoperation with Application to BCI Manipulation

TL;DR

This paper presents a shared-control framework combining computer vision, user intent inference, and arbitration to improve BCI-based robot teleoperation, enhancing performance, user comfort, and enabling complex tasks.

Contribution

It introduces a novel autonomous assistance system that balances human control and automation in BCI teleoperation, addressing noise and low-dimensional commands.

Findings

Significant performance improvements on rehabilitation benchmarks.

Shared assistance reduces perceived user difficulty.

Enables successful completion of complex, previously infeasible tasks.

Abstract

Robot teleoperation systems face a common set of challenges including latency, low-dimensional user commands, and asymmetric control inputs. User control with Brain-Computer Interfaces (BCIs) exacerbates these problems through especially noisy and erratic low-dimensional motion commands due to the difficulty in decoding neural activity. We introduce a general framework to address these challenges through a combination of computer vision, user intent inference, and arbitration between the human input and autonomous control schemes. Adjustable levels of assistance allow the system to balance the operator's capabilities and feelings of comfort and control while compensating for a task's difficulty. We present experimental results demonstrating significant performance improvement using the shared-control assistance framework on adapted rehabilitation benchmarks with two subjects implanted with intracortical brain-computer interfaces controlling a seven degree-of-freedom robotic manipulator as a prosthetic. Our results further indicate that shared assistance mitigates perceived user difficulty and even enables successful performance on previously infeasible tasks. We showcase the extensibility of our architecture with applications to quality-of-life tasks such as opening a door, pouring liquids from containers, and manipulation with novel objects in densely cluttered environments.