Evaluation of Human-Robot Interfaces based on 2D/3D Visual and Haptic Feedback for Aerial Manipulation

TL;DR

This study investigates how 3D visual and haptic feedback in aerial telemanipulation systems enhance operator dexterity and reduce workload, demonstrating the benefits of richer sensory information in complex manipulation tasks.

Contribution

It introduces a novel human-robot interface providing 3D mixed reality visual and haptic feedback and evaluates their effects on performance in aerial manipulation tasks.

Findings

Both 3D MR vision and haptic feedback improve dexterity.

Operator experience significantly influences task performance.

Rich sensory feedback reduces cognitive workload.

Abstract

Most telemanipulation systems for aerial robots provide the operator with only 2D screen visual information. The lack of richer information about the robot's status and environment can limit human awareness and, in turn, task performance. While the pilot's experience can often compensate for this reduced flow of information, providing richer feedback is expected to reduce the cognitive workload and offer a more intuitive experience overall. This work aims to understand the significance of providing additional pieces of information during aerial telemanipulation, namely (i) 3D immersive visual feedback about the robot's surroundings through mixed reality (MR) and (ii) 3D haptic feedback about the robot interaction with the environment. To do so, we developed a human-robot interface able to provide this information. First, we demonstrate its potential in a real-world manipulation task requiring sub-centimeter-level accuracy. Then, we evaluate the individual effect of MR vision and haptic feedback on both dexterity and workload through a human subjects study involving a virtual block transportation task. Results show that both 3D MR vision and haptic feedback improve the operator's dexterity in the considered teleoperated aerial interaction tasks. Nevertheless, pilot experience remains the most significant factor.