ScaleADFG: Affordance-based Dexterous Functional Grasping via Scalable Dataset

TL;DR

ScaleADFG introduces a scalable, automated dataset and a lightweight grasp network for dexterous robotic tool-use, improving generalization across object sizes and enabling zero-shot transfer to real-world scenarios.

Contribution

The paper presents a novel automated dataset construction pipeline and a simple grasp generation network that together enhance multi-scale grasping capabilities for robotic manipulation.

Findings

The dataset contains over 60,000 grasps across five object categories.

The grasp network achieves effective zero-shot transfer to real-world objects.

Experiments show improved grasp stability and diversity across varying object scales.

Abstract

Dexterous functional tool-use grasping is essential for effective robotic manipulation of tools. However, existing approaches face significant challenges in efficiently constructing large-scale datasets and ensuring generalizability to everyday object scales. These issues primarily arise from size mismatches between robotic and human hands, and the diversity in real-world object scales. To address these limitations, we propose the ScaleADFG framework, which consists of a fully automated dataset construction pipeline and a lightweight grasp generation network. Our dataset introduce an affordance-based algorithm to synthesize diverse tool-use grasp configurations without expert demonstrations, allowing flexible object-hand size ratios and enabling large robotic hands (compared to human hands) to grasp everyday objects effectively. Additionally, we leverage pre-trained models to generate extensive 3D assets and facilitate efficient retrieval of object affordances. Our dataset comprising five object categories, each containing over 1,000 unique shapes with 15 scale variations. After filtering, the dataset includes over 60,000 grasps for each 2 dexterous robotic hands. On top of this dataset, we train a lightweight, single-stage grasp generation network with a notably simple loss design, eliminating the need for post-refinement. This demonstrates the critical importance of large-scale datasets and multi-scale object variant for effective training. Extensive experiments in simulation and on real robot confirm that the ScaleADFG framework exhibits strong adaptability to objects of varying scales, enhancing functional grasp stability, diversity, and generalizability. Moreover, our network exhibits effective zero-shot transfer to real-world objects. Project page is available at https://sizhe-wang.github.io/ScaleADFG_webpage