Robot Tactile Gesture Recognition Based on Full-body Modular E-skin

TL;DR

This paper presents a modular electronic skin for robots that, combined with an equivariant graph neural network, enables accurate recognition of tactile gestures for intuitive human-robot interaction.

Contribution

The work introduces a novel modular E-skin design and a gesture recognition method based on equivariant graph neural networks for robots.

Findings

High accuracy in gesture classification

Real-time tactile perception capability

Effective human-robot interaction through tactile gestures

Abstract

With the development of robot electronic skin technology, various tactile sensors, enhanced by AI, are unlocking a new dimension of perception for robots. In this work, we explore how robots equipped with electronic skin can recognize tactile gestures and interpret them as human commands. We developed a modular robot E-skin, composed of multiple irregularly shaped skin patches, which can be assembled to cover the robot's body while capturing real-time pressure and pose data from thousands of sensing points. To process this information, we propose an equivariant graph neural network-based recognizer that efficiently and accurately classifies diverse tactile gestures, including poke, grab, stroke, and double-pat. By mapping the recognized gestures to predefined robot actions, we enable intuitive human-robot interaction purely through tactile input.