TRTM: Template-based Reconstruction and Target-oriented Manipulation of Crumpled Cloths

TL;DR

This paper introduces TRTM, a template-based system for reconstructing and manipulating crumpled cloths from a single top-view depth image, enabling efficient cloth handling despite occlusions and shape variations.

Contribution

The paper presents a novel template-based reconstruction method that explicitly models cloth meshes for improved manipulation, leveraging advances in human reconstruction techniques.

Findings

Effective reconstruction of daily cloths with different shapes and properties.

Enables target-oriented manipulation using explicit mesh representations.

System demonstrated on real-world cloth manipulation tasks.

Abstract

Precise reconstruction and manipulation of the crumpled cloths is challenging due to the high dimensionality of cloth models, as well as the limited observation at self-occluded regions. We leverage the recent progress in the field of single-view human reconstruction to template-based reconstruct crumpled cloths from their top-view depth observations only, with our proposed sim-real registration protocols. In contrast to previous implicit cloth representations, our reconstruction mesh explicitly describes the positions and visibilities of the entire cloth mesh vertices, enabling more efficient dual-arm and single-arm target-oriented manipulations. Experiments demonstrate that our TRTM system can be applied to daily cloths that have similar topologies as our template mesh, but with different shapes, sizes, patterns, and physical properties. Videos, datasets, pre-trained models, and code can be downloaded from our project website: https://wenbwa.github.io/TRTM/ .