Let Robots Feel Your Touch: Visuo-Tactile Cortical Alignment for Embodied Mirror Resonance

TL;DR

This paper introduces Mirror Touch Net, a computational framework that aligns visual and tactile representations in robots, enabling predictive and empathic responses to observed touch, inspired by neural principles of visuo-tactile resonance.

Contribution

It operationalizes cortical correspondence in robots through multi-level constraints, allowing cross-modal tactile prediction from visual input and extending to human touch observation.

Findings

Robots can predict tactile signals from RGB images with millimeter accuracy.

Alignment reshapes visual representations to match tactile manifolds, simplifying cross-modal mapping.

Framework enables reflexive responses to observed human touch.

Abstract

Observing touch on another's body can elicit corresponding tactile sensations in the observer, a phenomenon termed mirror touch that supports empathy and social perception. This visuo-tactile resonance is thought to rely on structural correspondence between visual and somatosensory cortices, yet robotic systems lack computational frameworks that instantiate this principle. Here we demonstrate that cortical correspondence can be operationalized to endow robots with mirror touch. We introduce Mirror Touch Net, which imposes semantic, distributional and geometric alignment between visual and tactile representations through multi-level constraints, enabling prediction of millimetre-scale tactile signals across 1,140 taxels on a robotic hand from RGB images. Manifold analysis reveals that these constraints reshape visual representations into geometry consistent with the tactile manifold, reducing the complexity of cross-modal mapping. Extending this alignment framework to cross-domain observations of human hands enables tactile prediction and reflexive responses to observed human touch. Our results link a neural principle of visuo-tactile resonance to robotic perception, providing an explainable route towards anticipatory touch and empathic human-robot interaction. Code is available at https://github.com/fun0515/Mirror-Touch-Net.