GelSight FlexiRay: Breaking Planar Limits by Harnessing Large Deformations for Flexible,Full-Coverage Multimodal Sensing

TL;DR

Gelsight FlexiRay is a multimodal tactile sensor integrated with soft robotic grippers, enabling large-area, high-resolution sensing during substantial deformations for safer and more versatile robotic interactions.

Contribution

The paper introduces Gelsight FlexiRay, a novel multimodal tactile sensor that maintains high performance under large deformations, surpassing existing compliant visual-tactile sensors in sensing area and robustness.

Findings

Achieves force measurement accuracy of 0.14 N.

Demonstrates 5 times larger deformation capacity than state-of-the-art sensors.

Provides robust tactile perception across diverse deformation states.

Abstract

The integration of tactile sensing into compliant soft robotic grippers offers a compelling pathway toward advanced robotic grasping and safer human-robot interactions. Visual-tactile sensors realize high-resolution, large-area tactile perception with affordable cameras. However, conventional visual-tactile sensors rely heavily on rigid forms, sacrificing finger compliance and sensing regions to achieve localized tactile feedback. Enabling seamless, large-area tactile sensing in soft grippers remains challenging, as deformations inherent to soft structures can obstruct the optical path and restrict the camera's field of view. To address these, we present Gelsight FlexiRay, a multimodal visual-tactile sensor designed for safe and compliant interactions with substantial structural deformation through integration with Finray Effect grippers. First, we adopt a multi-mirror configuration, which is systematically modeled and optimized based on the physical force-deformation characteristics of FRE grippers. Second, we enhanced Gelsight FlexiRay with human-like multimodal perception, including contact force and location, proprioception, temperature, texture, and slippage. Experiments demonstrate Gelsight FlexiRay's robust tactile performance across diverse deformation states, achieving a force measurement accuracy of 0.14 N and proprioceptive positioning accuracy of 0.19 mm. Compared with state of art compliant VTS, the FlexiRay demonstrates 5 times larger structural deformation under the same loads. Its expanded sensing area and ability to distinguish contact information and execute grasping and classification tasks highlights its potential for versatile, large-area multimodal tactile sensing integration within soft robotic systems. This work establishes a foundation for flexible, high-resolution tactile sensing in compliant robotic applications.