A Low-Cost Vision-Based Tactile Gripper with Pretraining Learning for Contact-Rich Manipulation

TL;DR

This paper introduces LVTG, a low-cost visuo-tactile gripper that combines enhanced sensing, durability, and a novel cross-modal learning approach to improve contact-rich manipulation performance.

Contribution

The work presents a new visuo-tactile gripper with a CLIP-inspired pretraining method for better perception and manipulation in contact-rich environments.

Findings

LVTG achieves higher success rates in manipulation tasks.

Enhanced tactile sensing area improves grasping of larger objects.

Pretraining with contrastive learning enhances policy performance.

Abstract

Robotic manipulation in contact-rich environments remains challenging, particularly when relying on conventional tactile sensors that suffer from limited sensing range, reliability, and cost-effectiveness. In this work, we present LVTG, a low-cost visuo-tactile gripper designed for stable, robust, and efficient physical interaction. Unlike existing visuo-tactile sensors, LVTG enables more effective and stable grasping of larger and heavier everyday objects, thanks to its enhanced tactile sensing area and greater opening angle. Its surface skin is made of highly wear-resistant material, significantly improving durability and extending operational lifespan. The integration of vision and tactile feedback allows LVTG to provide rich, high-fidelity sensory data, facilitating reliable perception during complex manipulation tasks. Furthermore, LVTG features a modular design that supports rapid maintenance and replacement. To effectively fuse vision and touch, We adopt a CLIP-inspired contrastive learning objective to align tactile embeddings with their corresponding visual observations, enabling a shared cross-modal representation space for visuo-tactile perception. This alignment improves the performance of an Action Chunking Transformer (ACT) policy in contact-rich manipulation, leading to more efficient data collection and more effective policy learning. Compared to the original ACT method, the proposed LVTG with pretraining achieves significantly higher success rates in manipulation tasks.