ZeroDexGrasp: Zero-Shot Task-Oriented Dexterous Grasp Synthesis with Prompt-Based Multi-Stage Semantic Reasoning

TL;DR

ZeroDexGrasp introduces a zero-shot framework combining multimodal language models and semantic reasoning to generate task-specific dexterous grasps for unseen objects, reducing reliance on labeled data.

Contribution

It presents a novel zero-shot grasp synthesis method using prompt-based multi-stage reasoning and contact-guided optimization, enabling generalization across diverse objects and tasks.

Findings

Achieves high-quality zero-shot grasping on unseen objects.

Demonstrates effective task-object semantic alignment.

Outperforms existing methods in generalization and flexibility.

Abstract

Task-oriented dexterous grasping holds broad application prospects in robotic manipulation and human-object interaction. However, most existing methods still struggle to generalize across diverse objects and task instructions, as they heavily rely on costly labeled data to ensure task-specific semantic alignment. In this study, we propose \textbf{ZeroDexGrasp}, a zero-shot task-oriented dexterous grasp synthesis framework integrating Multimodal Large Language Models with grasp refinement to generate human-like grasp poses that are well aligned with specific task objectives and object affordances. Specifically, ZeroDexGrasp employs prompt-based multi-stage semantic reasoning to infer initial grasp configurations and object contact information from task and object semantics, then exploits contact-guided grasp optimization to refine these poses for physical feasibility and task alignment. Experimental results demonstrate that ZeroDexGrasp enables high-quality zero-shot dexterous grasping on diverse unseen object categories and complex task requirements, advancing toward more generalizable and intelligent robotic grasping.