StereoTacTip: Vision-based Tactile Sensing with Biomimetic Skin-Marker Arrangements

TL;DR

This paper introduces StereoTacTip, a vision-based tactile sensor that uses biomimetic skin markers and novel algorithms for accurate 3D surface reconstruction, addressing challenges posed by complex skin-marker arrangements.

Contribution

The paper presents a new stereo vision-based tactile sensor with innovative marker matching, depth correction, and surface modeling techniques for improved geometry reconstruction.

Findings

Effective stereo marker matching with Delaunay-Triangulation-Ring-Coding

Refractive depth correction improves accuracy

Successful reconstruction of complex 3D terrains

Abstract

Vision-Based Tactile Sensors (VBTSs) stand out for their superior performance due to their high-information content output. Recently, marker-based VBTSs have been shown to give accurate geometry reconstruction when using stereo cameras. \uhl{However, many marker-based VBTSs use complex biomimetic skin-marker arrangements, which presents issues for the geometric reconstruction of the skin surface from the markers}. Here we investigate how the marker-based skin morphology affects stereo vision-based tactile sensing, using a novel VBTS called the StereoTacTip. To achieve accurate geometry reconstruction, we introduce: (i) stereo marker matching and tracking using a novel Delaunay-Triangulation-Ring-Coding algorithm; (ii) a refractive depth correction model that corrects the depth distortion caused by refraction in the internal media; (iii) a skin surface correction model from the marker positions, relying on an inverse calculation of normals to the skin surface; and (iv)~methods for geometry reconstruction over multiple contacts. To demonstrate these findings, we reconstruct topographic terrains on a large 3D map. Even though contributions (i) and (ii) were developed for biomimetic markers, they should improve the performance of all marker-based VBTSs. Overall, this work illustrates that a thorough understanding and evaluation of the morphologically-complex skin and marker-based tactile sensor principles are crucial for obtaining accurate geometric information.