Differentiable Particles for General-Purpose Deformable Object Manipulation

TL;DR

This paper introduces Differentiable Particles (DiPac), a versatile algorithm for deformable object manipulation that combines learning, planning, and differentiable simulation to handle various object types and improve transferability.

Contribution

DiPac is a novel particle-based deformable object manipulation method that integrates differentiable dynamics with planning and learning for general-purpose use.

Findings

DiPac effectively manipulates diverse deformable objects in simulation and real-world tests.

It outperforms pure planning or learning methods in manipulation tasks.

DiPac demonstrates robust transfer of policies across different object properties.

Abstract

Deformable object manipulation is a long-standing challenge in robotics. While existing approaches often focus narrowly on a specific type of object, we seek a general-purpose algorithm, capable of manipulating many different types of objects: beans, rope, cloth, liquid, . . . . One key difficulty is a suitable representation, rich enough to capture object shape, dynamics for manipulation and yet simple enough to be acquired effectively from sensor data. Specifically, we propose Differentiable Particles (DiPac), a new algorithm for deformable object manipulation. DiPac represents a deformable object as a set of particles and uses a differentiable particle dynamics simulator to reason about robot manipulation. To find the best manipulation action, DiPac combines learning, planning, and trajectory optimization through differentiable trajectory tree optimization. Differentiable dynamics provides significant benefits and enable DiPac to (i) estimate the dynamics parameters efficiently, thereby narrowing the sim-to-real gap, and (ii) choose the best action by backpropagating the gradient along sampled trajectories. Both simulation and real-robot experiments show promising results. DiPac handles a variety of object types. By combining planning and learning, DiPac outperforms both pure model-based planning methods and pure data-driven learning methods. In addition, DiPac is robust and adapts to changes in dynamics, thereby enabling the transfer of an expert policy from one object to another with different physical properties, e.g., from a rigid rod to a deformable rope.