Transformer Driven Visual Servoing for Fabric Texture Matching Using Dual-Arm Manipulator
Fuyuki Tokuda, Akira Seino, Akinari Kobayashi, Kai Tang, Kazuhiro Kosuge

TL;DR
This paper introduces a novel transformer-based visual servoing method for dual-arm robots to accurately align and place fabric textures, leveraging synthetic training for real-world application without prior texture data.
Contribution
It presents a new control scheme combining transformer-driven visual servoing with dual-arm impedance control and a novel attention module, enabling zero-shot fabric texture matching.
Findings
Accurate fabric alignment demonstrated in real-world tests.
Synthetic training enables zero-shot deployment.
Enhanced pose prediction accuracy with DEAM.
Abstract
In this paper, we propose a method to align and place a fabric piece on top of another using a dual-arm manipulator and a grayscale camera, so that their surface textures are accurately matched. We propose a novel control scheme that combines Transformer-driven visual servoing with dualarm impedance control. This approach enables the system to simultaneously control the pose of the fabric piece and place it onto the underlying one while applying tension to keep the fabric piece flat. Our transformer-based network incorporates pretrained backbones and a newly introduced Difference Extraction Attention Module (DEAM), which significantly enhances pose difference prediction accuracy. Trained entirely on synthetic images generated using rendering software, the network enables zero-shot deployment in real-world scenarios without requiring prior training on specific fabric textures. Real-world…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · 3D Shape Modeling and Analysis · Robot Manipulation and Learning
