DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping

TL;DR

DexGraspVLA introduces a hierarchical vision-language-action framework that significantly improves general dexterous grasping in complex, unseen environments by leveraging foundation models and diffusion-based control.

Contribution

It presents a novel hierarchical framework combining pre-trained vision-language models and diffusion-based controllers for robust, generalizable dexterous grasping beyond prior restrictive assumptions.

Findings

Achieves over 90% success rate in unseen cluttered scenes.

Demonstrates robustness to adversarial objects and disturbances.

Enables free-form long-horizon prompt execution.

Abstract

Dexterous grasping remains a fundamental yet challenging problem in robotics. A general-purpose robot must be capable of grasping diverse objects in arbitrary scenarios. However, existing research typically relies on restrictive assumptions, such as single-object settings or limited environments, showing constrained generalization. We present DexGraspVLA, a hierarchical framework for robust generalization in language-guided general dexterous grasping and beyond. It utilizes a pre-trained Vision-Language model as the high-level planner and learns a diffusion-based low-level Action controller. The key insight to achieve generalization lies in iteratively transforming diverse language and visual inputs into domain-invariant representations via foundation models, where imitation learning can be effectively applied due to the alleviation of domain shift. Notably, our method achieves a 90+% dexterous grasping success rate under thousands of challenging unseen cluttered scenes. Empirical analysis confirms the consistency of internal model behavior across environmental variations, validating our design. DexGraspVLA also, for the first time, simultaneously demonstrates free-form long-horizon prompt execution, robustness to adversarial objects and human disturbance, and failure recovery. Extended application to nonprehensile grasping further proves its generality. Project website: https://dexgraspvla.github.io.