PartDexTOG: Generating Dexterous Task-Oriented Grasping via Language-driven Part Analysis

TL;DR

This paper introduces PartDexTOG, a novel approach that leverages language-driven part analysis and diffusion models to generate dexterous, task-oriented grasps for robotic manipulation, significantly improving performance and generality.

Contribution

The paper presents a new method combining language-driven part analysis with diffusion models to enhance dexterous task-oriented grasping, addressing limitations of previous approaches.

Findings

Achieves top performance on OakInk-shape dataset

Improves Penetration Volume, Grasp Displace, and P-FID metrics significantly

Demonstrates strong generality across novel categories and tasks

Abstract

Task-oriented grasping is a crucial yet challenging task in robotic manipulation. Despite the recent progress, few existing methods address task-oriented grasping with dexterous hands. Dexterous hands provide better precision and versatility, enabling robots to perform task-oriented grasping more effectively. In this paper, we argue that part analysis can enhance dexterous grasping by providing detailed information about the object's functionality. We propose PartDexTOG, a method that generates dexterous task-oriented grasps via language-driven part analysis. Taking a 3D object and a manipulation task represented by language as input, the method first generates the category-level and part-level grasp descriptions w.r.t the manipulation task by LLMs. Then, a category-part conditional diffusion model is developed to generate a dexterous grasp for each part, respectively, based on the generated descriptions. To select the most plausible combination of grasp and corresponding part from the generated ones, we propose a measure of geometric consistency between grasp and part. We show that our method greatly benefits from the open-world knowledge reasoning on object parts by LLMs, which naturally facilitates the learning of grasp generation on objects with different geometry and for different manipulation tasks. Our method ranks top on the OakInk-shape dataset over all previous methods, improving the Penetration Volume, the Grasp Displace, and the P-FID over the state-of-the-art by $3.58\%$, $2.87\%$, and $41.43\%$, respectively. Notably, it demonstrates good generality in handling novel categories and tasks.