Imagine2touch: Predictive Tactile Sensing for Robotic Manipulation using Efficient Low-Dimensional Signals

TL;DR

Imagine2touch enables robots to predict tactile signals from visual cues, improving object recognition by leveraging low-dimensional touch data, thus mimicking human touch perception in robotic manipulation.

Contribution

This work introduces a novel predictive tactile sensing framework using low-dimensional signals, trained on a new dataset collected with ReSkin sensors, and demonstrates its application in object recognition.

Findings

Achieved 58% object recognition accuracy after ten touches.

Outperformed proprioception baseline in tactile-based object recognition.

Validated on out-of-distribution tool data.

Abstract

Humans seemingly incorporate potential touch signals in their perception. Our goal is to equip robots with a similar capability, which we term Imagine2touch. Imagine2touch aims to predict the expected touch signal based on a visual patch representing the area to be touched. We use ReSkin, an inexpensive and compact touch sensor to collect the required dataset through random touching of five basic geometric shapes, and one tool. We train Imagine2touch on two out of those shapes and validate it on the ood. tool. We demonstrate the efficacy of Imagine2touch through its application to the downstream task of object recognition. In this task, we evaluate Imagine2touch performance in two experiments, together comprising 5 out of training distribution objects. Imagine2touch achieves an object recognition accuracy of 58% after ten touches per object, surpassing a proprioception baseline.