Learning Bimanual Cloth Manipulation with Vision-based Tactile Sensing via Single Robotic Arm

TL;DR

This paper introduces Touch G.O.G., a compact vision-based tactile system enabling a single robotic arm to perform bimanual cloth manipulation with high accuracy and reliability, reducing hardware complexity.

Contribution

The paper presents a novel tactile gripper, a vision transformer-based perception pipeline, and a synthetic data generator for effective single-arm cloth manipulation.

Findings

96% accuracy in cloth part classification

Sub-millimeter edge localization accuracy

Reliable cloth unfolding in real-world scenarios

Abstract

Robotic cloth manipulation remains challenging due to the high-dimensional state space of fabrics, their deformable nature, and frequent occlusions that limit vision-based sensing. Although dual-arm systems can mitigate some of these issues, they increase hardware and control complexity. This paper presents Touch G.O.G., a compact vision-based tactile gripper and perception/control framework for single-arm bimanual cloth manipulation. The proposed framework combines three key components: (1) a novel gripper design and control strategy for in-gripper cloth sliding with a single robot arm, (2) a Vision Foundation Model-backboned Vision Transformer pipeline for cloth part classification (PC-Net) and edge pose estimation (PE-Net) using real and synthetic tactile images, and (3) an encoder-decoder synthetic data generator (SD-Net) that reduces manual annotation by producing high-fidelity tactile images. Experiments show 96% accuracy in distinguishing edges, corners, interior regions, and grasp failures, together with sub-millimeter edge localization and 4.5{\deg} orientation error. Real-world results demonstrate reliable cloth unfolding, even for crumpled fabrics, using only a single robotic arm. These results highlight Touch G.O.G. as a compact and cost-effective solution for deformable object manipulation.