Dual-modal Tactile E-skin: Enabling Bidirectional Human-Robot Interaction via Integrated Tactile Perception and Feedback

TL;DR

This paper introduces a dual-modal electronic skin that combines magnetic tactile sensing and vibration feedback, enabling bidirectional tactile communication to improve human-robot interaction.

Contribution

The work presents a novel dual-modal e-skin integrating tactile perception and haptic feedback with deep learning for accurate sensing and programmable feedback.

Findings

Achieved accurate tactile perception using deep learning models.

Demonstrated bidirectional tactile communication for object recognition.

Enabled fine-weighing operations through the e-skin.

Abstract

To foster an immersive and natural human-robot interaction, the implementation of tactile perception and feedback becomes imperative, effectively bridging the conventional sensory gap. In this paper, we propose a dual-modal electronic skin (e-skin) that integrates magnetic tactile sensing and vibration feedback for enhanced human-robot interaction. The dual-modal tactile e-skin offers multi-functional tactile sensing and programmable haptic feedback, underpinned by a layered structure comprised of flexible magnetic films, soft silicone, a Hall sensor and actuator array, and a microcontroller unit. The e-skin captures the magnetic field changes caused by subtle deformations through Hall sensors, employing deep learning for accurate tactile perception. Simultaneously, the actuator array generates mechanical vibrations to facilitate haptic feedback, delivering diverse mechanical stimuli. Notably, the dual-modal e-skin is capable of transmitting tactile information bidirectionally, enabling object recognition and fine-weighing operations. This bidirectional tactile interaction framework will enhance the immersion and efficiency of interactions between humans and robots.