Learning Fabric Manipulation in the Real World with Human Videos
Robert Lee, Jad Abou-Chakra, Fangyi Zhang, Peter Corke

TL;DR
This paper presents a method for learning fabric manipulation directly from human videos, enabling robots to perform tasks like folding with minimal data and no additional robot training, bridging the sim-to-real gap.
Contribution
It introduces a natural, efficient data collection pipeline from human demonstrations and demonstrates real-world fabric folding with minimal demonstrations and no robot data collection.
Findings
Robust fabric folding from crumpled states
Effective learning from few human demonstrations
No robot data collection needed for deployment
Abstract
Fabric manipulation is a long-standing challenge in robotics due to the enormous state space and complex dynamics. Learning approaches stand out as promising for this domain as they allow us to learn behaviours directly from data. Most prior methods however rely heavily on simulation, which is still limited by the large sim-to-real gap of deformable objects or rely on large datasets. A promising alternative is to learn fabric manipulation directly from watching humans perform the task. In this work, we explore how demonstrations for fabric manipulation tasks can be collected directly by humans, providing an extremely natural and fast data collection pipeline. Then, using only a handful of such demonstrations, we show how a pick-and-place policy can be learned and deployed on a real robot, without any robot data collection at all. We demonstrate our approach on a fabric folding task,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobot Manipulation and Learning · Modular Robots and Swarm Intelligence · Innovations in Concrete and Construction Materials
