GenDOM: Generalizable One-shot Deformable Object Manipulation with Parameter-Aware Policy

TL;DR

GenDOM enables one-shot deformable object manipulation by conditioning policies on object parameters and estimating these parameters from a single demonstration, significantly reducing training data requirements.

Contribution

The paper introduces a parameter-aware policy framework that generalizes to new deformable objects with only one real-world demonstration, trained on diverse simulations.

Findings

Achieves 62% in-domain rope manipulation improvement

Attains 50% cloth manipulation improvement in real-world tests

Outperforms baseline methods in both simulated and real environments

Abstract

Due to the inherent uncertainty in their deformability during motion, previous methods in deformable object manipulation, such as rope and cloth, often required hundreds of real-world demonstrations to train a manipulation policy for each object, which hinders their applications in our ever-changing world. To address this issue, we introduce GenDOM, a framework that allows the manipulation policy to handle different deformable objects with only a single real-world demonstration. To achieve this, we augment the policy by conditioning it on deformable object parameters and training it with a diverse range of simulated deformable objects so that the policy can adjust actions based on different object parameters. At the time of inference, given a new object, GenDOM can estimate the deformable object parameters with only a single real-world demonstration by minimizing the disparity between the grid density of point clouds of real-world demonstrations and simulations in a differentiable physics simulator. Empirical validations on both simulated and real-world object manipulation setups clearly show that our method can manipulate different objects with a single demonstration and significantly outperforms the baseline in both environments (a 62% improvement for in-domain ropes and a 15% improvement for out-of-distribution ropes in simulation, as well as a 26% improvement for ropes and a 50% improvement for cloths in the real world), demonstrating the effectiveness of our approach in one-shot deformable object manipulation.