SpaceDex: Generalizable Dexterous Grasping in Tiered Workspaces

TL;DR

SpaceDex introduces a hierarchical framework combining vision-language planning and feature separation to improve dexterous grasping in constrained 3D environments, achieving significant success rate improvements over baseline methods.

Contribution

The paper presents a novel hierarchical approach with an arm-hand feature separation network and multi-view perception for generalizable grasping in tiered workspaces.

Findings

Achieved 63.0% success rate in real-world trials with unseen objects.

Outperformed a strong tabletop baseline with 39.0% success rate.

Demonstrated robustness to partial observability and off-nominal contacts.

Abstract

Generalizable grasping with high-degree-of-freedom (DoF) dexterous hands remains challenging in tiered workspaces, where occlusion, narrow clearances, and height-dependent constraints are substantially stronger than in open tabletop scenes. Most existing methods are evaluated in relatively unoccluded settings and typically do not explicitly model the distinct control requirements of arm navigation and hand articulation under spatial constraints. We present SpaceDex, a hierarchical framework for dexterous manipulation in constrained 3D environments. At the high level, a Vision-Language Model (VLM) planner parses user intent, reasons about occlusion and height relations across multiple camera views, and generates target bounding boxes for zero-shot segmentation and mask tracking. This stage provides structured spatial guidance for downstream control instead of relying on single-view target selection. At the low level, we introduce an arm-hand Feature Separation Network that decouples global trajectory control for the arm from geometry-aware grasp mode selection for the hand, reducing feature interference between reaching and grasping objectives. The controller further integrates multi-view perception, fingertip tactile sensing, and a small set of recovery demonstrations to improve robustness to partial observability and off-nominal contacts. In 100 real-world trials involving over 30 unseen objects across four categories, SpaceDex achieves a 63.0\% success rate, compared with 39.0\% for a strong tabletop baseline. These results indicate that combining hierarchical spatial planning with arm-hand representation decoupling improves dexterous grasping performance in spatially constrained environments.