Alternative Modes of Interaction in Proximal Human-in-the-Loop Operation of Robots

TL;DR

This paper explores innovative human-robot interaction methods using augmented reality and EEG feedback to improve safety, efficiency, and collaboration in proximal, human-in-the-loop robotic tasks.

Contribution

It introduces a system combining AR cues and EEG monitoring to enhance communication and adaptive response in human-robot teams during collaborative tasks.

Findings

AR techniques effectively project robot intentions.

EEG feedback enables adaptive robot behavior.

System demonstrates real-time multimodal interaction.

Abstract

Ambiguity and noise in natural language instructions create a significant barrier towards adopting autonomous systems into safety critical workflows involving humans and machines. In this paper, we propose to build on recent advances in electrophysiological monitoring methods and augmented reality technologies, to develop alternative modes of communication between humans and robots involved in large-scale proximal collaborative tasks. We will first introduce augmented reality techniques for projecting a robot's intentions to its human teammate, who can interact with these cues to engage in real-time collaborative plan execution with the robot. We will then look at how electroencephalographic (EEG) feedback can be used to monitor human response to both discrete events, as well as longer term affective states while execution of a plan. These signals can be used by a learning agent, a.k.a an affective robot, to modify its policy. We will present an end-to-end system capable of demonstrating these modalities of interaction. We hope that the proposed system will inspire research in augmenting human-robot interactions with alternative forms of communications in the interests of safety, productivity, and fluency of teaming, particularly in engineered settings such as the factory floor or the assembly line in the manufacturing industry where the use of such wearables can be enforced.