BLaDA: Bridging Language to Functional Dexterous Actions within 3DGS Fields

TL;DR

BLaDA is an interpretable zero-shot framework that grounds open-vocabulary instructions for functional dexterous manipulation in unstructured environments, integrating semantic understanding, spatial reasoning, and pose execution.

Contribution

It introduces a novel structured reasoning chain and modules for pose-consistent spatial reasoning and control, enabling zero-shot functional manipulation without predefined labels.

Findings

Outperforms existing methods in affordance grounding precision.

Achieves higher success rates in diverse functional manipulation tasks.

Demonstrates effective open-vocabulary instruction grounding in complex benchmarks.

Abstract

In unstructured environments, functional dexterous grasping calls for the tight integration of semantic understanding, precise 3D functional localization, and physically interpretable execution. Modular hierarchical methods are more controllable and interpretable than end-to-end VLA approaches, but existing ones still rely on predefined affordance labels and lack the tight semantic--pose coupling needed for functional dexterous manipulation. To address this, we propose BLaDA (Bridging Language to Dexterous Actions in 3DGS fields), an interpretable zero-shot framework that grounds open-vocabulary instructions as perceptual and control constraints for functional dexterous manipulation. BLaDA establishes an interpretable reasoning chain by first parsing natural language into a structured sextuple of manipulation constraints via a Knowledge-guided Language Parsing (KLP) module. To achieve pose-consistent spatial reasoning, we introduce the Triangular Functional Point Localization (TriLocation) module, which utilizes 3D Gaussian Splatting as a continuous scene representation and identifies functional regions under triangular geometric constraints. Finally, the 3D Keypoint Grasp Matrix Transformation Execution (KGT3D+) module decodes these semantic-geometric constraints into physically plausible wrist poses and finger-level commands. Extensive experiments on complex benchmarks demonstrate that BLaDA significantly outperforms existing methods in both affordance grounding precision and the success rate of functional manipulation across diverse categories and tasks. Code will be publicly available at https://github.com/PopeyePxx/BLaDA.