CobotTouch: AR-based Interface with Fingertip-worn Tactile Display for Immersive Operation/Control of Collaborative Robots

TL;DR

CobotTouch introduces an AR-based fingertip haptic interface with gesture recognition for intuitive collaborative robot control, reducing mental load and enhancing user experience.

Contribution

This work presents a novel AR interface with tactile feedback for robot control, combining gesture recognition and fingertip haptics to improve intuitiveness and reduce cognitive load.

Findings

75.25% tactile pattern recognition rate

Low mental and temporal demands in user interaction

High perceived intuitiveness of the system

Abstract

Complex robotic tasks require human collaboration to benefit from their high dexterity. Frequent human-robot interaction is mentally demanding and time-consuming. Intuitive and easy-to-use robot control interfaces reduce the negative influence on workers, especially inexperienced users. In this paper, we present CobotTouch, a novel intuitive robot control interface with fingertip haptic feedback. The proposed interface consists of a projected Graphical User Interface on the robotic arm to control the position of the robot end-effector based on gesture recognition, and a wearable haptic interface to deliver tactile feedback on the user's fingertips. We evaluated the user's perception of the designed tactile patterns presented by the haptic interface and the intuitiveness of the proposed system for robot control in a use case. The results revealed a high average recognition rate of 75.25\% for the tactile patterns. An average NASA Task Load Index (TLX) indicated small mental and temporal demands proving a high level of the intuitiveness of CobotTouch for interaction with collaborative robots.