PinchBot: Long-Horizon Deformable Manipulation with Guided Diffusion Policy

TL;DR

PinchBot is a goal-conditioned diffusion policy that enables a robot to perform long-horizon deformable manipulation tasks like pottery creation using pinch actions, leveraging 3D embeddings and collision constraints.

Contribution

This work introduces PinchBot, a novel diffusion policy model for deformable manipulation with long-horizon goals, combining 3D embeddings and collision-aware actions.

Findings

Successfully creates simple pottery shapes

Handles multi-modal, long-horizon deformable tasks

Demonstrates effective goal-conditioned manipulation

Abstract

Pottery creation is a complicated art form that requires dexterous, precise and delicate actions to slowly morph a block of clay to a meaningful, and often useful 3D goal shape. In this work, we aim to create a robotic system that can create simple pottery goals with only pinch-based actions. This pinch pottery task allows us to explore the challenges of a highly multi-modal and long-horizon deformable manipulation task. To this end, we present PinchBot, a goal-conditioned diffusion policy model that when combined with pre-trained 3D point cloud embeddings, task progress prediction and collision-constrained action projection, is able to successfully create a variety of simple pottery goals. For experimental videos and access to the demonstration dataset, please visit our project website: https://sites.google.com/andrew.cmu.edu/pinchbot/home.