Proximity and Visuotactile Point Cloud Fusion for Contact Patches in Extreme Deformation

TL;DR

This paper introduces a novel fusion algorithm combining proximity and visuotactile point clouds to accurately segment contact patches in highly deformable objects, outperforming existing methods across various strain levels.

Contribution

The work presents a deformation-independent fusion algorithm leveraging high-resolution sensing modalities, enabling robust contact patch segmentation in extreme deformation regimes.

Findings

Outperforms baseline methods with RMSE under 2.8 mm across strain levels

Effective in applications like pose estimation and closed-loop control

Works with various membrane stiffness and deformation conditions

Abstract

Visuotactile sensors are a popular tactile sensing strategy due to high-fidelity estimates of local object geometry. However, existing algorithms for processing raw sensor inputs to useful intermediate signals such as contact patches struggle in high-deformation regimes. This is due to physical constraints imposed by sensor hardware and small-deformation assumptions used by mechanics-based models. In this work, we propose a fusion algorithm for proximity and visuotactile point clouds for contact patch segmentation, entirely independent from membrane mechanics. This algorithm exploits the synchronous, high spatial resolution proximity and visuotactile modalities enabled by an extremely deformable, selectively transmissive soft membrane, which uses visible light for visuotactile sensing and infrared light for proximity depth. We evaluate our contact patch algorithm in low (10%), medium (60%), and high (100%+) strain states. We compare our method against three baselines: proximity-only, tactile-only, and a first principles mechanics model. Our approach outperforms all baselines with an average RMSE under 2.8 mm of the contact patch geometry across all strain ranges. We demonstrate our contact patch algorithm in four applications: varied stiffness membranes, torque and shear-induced wrinkling, closed loop control, and pose estimation.