Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation

TL;DR

This paper introduces GLIDE, a planning-guided diffusion policy learning method that leverages model-based planning and synthetic demonstrations to enable generalizable contact-rich bimanual manipulation in simulation and real-world scenarios.

Contribution

The paper presents a novel approach combining planning, diffusion policies, and robust features to improve generalization in complex bimanual manipulation tasks.

Findings

Effective manipulation of diverse objects demonstrated in simulation.

Successful transfer of policies from simulation to real-world robots.

Enhanced robustness and generalization through feature and data augmentation.

Abstract

Contact-rich bimanual manipulation involves precise coordination of two arms to change object states through strategically selected contacts and motions. Due to the inherent complexity of these tasks, acquiring sufficient demonstration data and training policies that generalize to unseen scenarios remain a largely unresolved challenge. Building on recent advances in planning through contacts, we introduce Generalizable Planning-Guided Diffusion Policy Learning (GLIDE), an approach that effectively learns to solve contact-rich bimanual manipulation tasks by leveraging model-based motion planners to generate demonstration data in high-fidelity physics simulation. Through efficient planning in randomized environments, our approach generates large-scale and high-quality synthetic motion trajectories for tasks involving diverse objects and transformations. We then train a task-conditioned diffusion policy via behavior cloning using these demonstrations. To tackle the sim-to-real gap, we propose a set of essential design options in feature extraction, task representation, action prediction, and data augmentation that enable learning robust prediction of smooth action sequences and generalization to unseen scenarios. Through experiments in both simulation and the real world, we demonstrate that our approach can enable a bimanual robotic system to effectively manipulate objects of diverse geometries, dimensions, and physical properties. Website: https://glide-manip.github.io/