FoldNet: Learning Generalizable Closed-Loop Policy for Garment Folding via Keypoint-Driven Asset and Demonstration Synthesis

TL;DR

This paper introduces FoldNet, a novel approach for robotic garment folding that uses synthetic data, keypoint-driven demonstrations, and closed-loop imitation learning to achieve high success rates in real-world tasks.

Contribution

The paper presents a new synthetic dataset, a keypoint-based demonstration generation method, and KG-DAgger for improved robustness in garment folding policies.

Findings

Real-world success rate increased by 25% with KG-DAgger.

Model trained on 15K trajectories achieves 75% success rate.

Framework validated in both simulation and real-world environments.

Abstract

Due to the deformability of garments, generating a large amount of high-quality data for robotic garment manipulation tasks is highly challenging. In this paper, we present a synthetic garment dataset that can be used for robotic garment folding. We begin by constructing geometric garment templates based on keypoints and applying generative models to generate realistic texture patterns. Leveraging these keypoint annotations, we generate folding demonstrations in simulation and train folding policies via closed-loop imitation learning. To improve robustness, we propose KG-DAgger, which uses a keypoint-based strategy to generate demonstration data for recovering from failures. KG-DAgger significantly improves the model performance, boosting the real-world success rate by 25\%. After training with 15K trajectories (about 2M image-action pairs), the model achieves a 75\% success rate in the real world. Experiments in both simulation and real-world settings validate the effectiveness of our proposed framework.