Tactile Gesture Recognition with Built-in Joint Sensors for Industrial Robots

TL;DR

This paper demonstrates that deep learning models using only a robot's built-in joint sensors can accurately recognize gestures, eliminating the need for external sensors and advancing cost-effective human-robot collaboration.

Contribution

It introduces a novel approach using spectrogram-based data representations with CNNs for tactile gesture recognition relying solely on joint sensors.

Findings

Spectrogram representations significantly improve recognition accuracy.

Models generalize better to new robot poses with spectrogram data.

Achieved over 95% accuracy in contact detection and gesture classification.

Abstract

While gesture recognition using vision or robot skins is an active research area in Human-Robot Collaboration (HRC), this paper explores deep learning methods relying solely on a robot's built-in joint sensors, eliminating the need for external sensors. We evaluated various convolutional neural network (CNN) architectures and collected two datasets to study the impact of data representation and model architecture on the recognition accuracy. Our results show that spectrogram-based representations significantly improve accuracy, while model architecture plays a smaller role. We also tested generalization to new robot poses, where spectrogram-based models performed better. Implemented on a Franka Emika Research robot, two of our methods, STFT2DCNN and STT3DCNN, achieved over 95% accuracy in contact detection and gesture classification. These findings demonstrate the feasibility of external-sensor-free tactile recognition and promote further research toward cost-effective, scalable solutions for HRC.