Building a Library of Tactile Skills Based on FingerVision

TL;DR

This paper enhances the FingerVision tactile sensor by utilizing local skin deformation data for improved control and learning, enabling complex manipulation tasks like texture discrimination and precise force application.

Contribution

It introduces a novel approach using local marker deviations for tactile feedback, expanding FingerVision's applicability to challenging manipulation tasks.

Findings

Able to distinguish textures and viscosities during stirring.

Mapped skin deformation to applied force accurately.

Successfully performed complex assembly tasks with tactile feedback.

Abstract

Camera-based tactile sensors are emerging as a promising inexpensive solution for tactile-enhanced manipulation tasks. A recently introduced FingerVision sensor was shown capable of generating reliable signals for force estimation, object pose estimation, and slip detection. In this paper, we build upon the FingerVision design, improving already existing control algorithms, and, more importantly, expanding its range of applicability to more challenging tasks by utilizing raw skin deformation data for control. In contrast to previous approaches that rely on the average deformation of the whole sensor surface, we directly employ local deviations of each spherical marker immersed in the silicone body of the sensor for feedback control and as input to learning tasks. We show that with such input, substances of varying texture and viscosity can be distinguished on the basis of tactile sensations evoked while stirring them. As another application, we learn a mapping between skin deformation and force applied to an object. To demonstrate the full range of capabilities of the proposed controllers, we deploy them in a challenging architectural assembly task that involves inserting a load-bearing element underneath a bendable plate at the point of maximum load.