Dexterous Functional Pre-Grasp Manipulation with Diffusion Policy

TL;DR

This paper introduces a novel learning framework combining teacher-student training, mixture-of-experts, and diffusion policies to enable dexterous pre-grasp manipulation across diverse objects, achieving a success rate of 72.6%.

Contribution

It presents a new approach integrating mutual rewards, expert mixtures, and diffusion models for versatile pre-grasp manipulation, advancing generalization and control in robotic hand tasks.

Findings

Achieved 72.6% success rate across 30+ object categories.

Effectively learned diverse manipulation policies.

Demonstrated improved generalization in pre-grasp tasks.

Abstract

In real-world scenarios, objects often require repositioning and reorientation before they can be grasped, a process known as pre-grasp manipulation. Learning universal dexterous functional pre-grasp manipulation requires precise control over the relative position, orientation, and contact between the hand and object while generalizing to diverse dynamic scenarios with varying objects and goal poses. To address this challenge, we propose a teacher-student learning approach that utilizes a novel mutual reward, incentivizing agents to optimize three key criteria jointly. Additionally, we introduce a pipeline that employs a mixture-of-experts strategy to learn diverse manipulation policies, followed by a diffusion policy to capture complex action distributions from these experts. Our method achieves a success rate of 72.6\% across more than 30 object categories by leveraging extrinsic dexterity and adjusting from feedback.