UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence

TL;DR

This paper introduces UniGarmentManip, a unified framework that leverages dense visual correspondence to enable category-level garment manipulation, improving generalization across diverse garment types and reducing reliance on human annotations.

Contribution

The paper proposes a self-supervised method to learn topological dense visual correspondence among garments, facilitating versatile manipulation policies with minimal demonstrations.

Findings

Effective across three garment categories and multiple tasks

Works with one or two robotic arms and various garment states

Reduces need for task-specific policies and extensive annotations

Abstract

Garment manipulation (e.g., unfolding, folding and hanging clothes) is essential for future robots to accomplish home-assistant tasks, while highly challenging due to the diversity of garment configurations, geometries and deformations. Although able to manipulate similar shaped garments in a certain task, previous works mostly have to design different policies for different tasks, could not generalize to garments with diverse geometries, and often rely heavily on human-annotated data. In this paper, we leverage the property that, garments in a certain category have similar structures, and then learn the topological dense (point-level) visual correspondence among garments in the category level with different deformations in the self-supervised manner. The topological correspondence can be easily adapted to the functional correspondence to guide the manipulation policies for various downstream tasks, within only one or few-shot demonstrations. Experiments over garments in 3 different categories on 3 representative tasks in diverse scenarios, using one or two arms, taking one or more steps, inputting flat or messy garments, demonstrate the effectiveness of our proposed method. Project page: https://warshallrho.github.io/unigarmentmanip.