UniGraspTransformer: Simplified Policy Distillation for Scalable Dexterous Robotic Grasping

TL;DR

UniGraspTransformer is a scalable, Transformer-based policy network for dexterous robotic grasping that simplifies training and improves performance across diverse objects and real-world scenarios.

Contribution

It introduces a universal Transformer model trained via policy distillation, simplifying training and enhancing scalability for robotic grasping tasks.

Findings

Achieves higher success rates than prior methods on various object categories.

Handles thousands of objects with diverse poses effectively.

Generalizes well to real-world, vision-based grasping scenarios.

Abstract

We introduce UniGraspTransformer, a universal Transformer-based network for dexterous robotic grasping that simplifies training while enhancing scalability and performance. Unlike prior methods such as UniDexGrasp++, which require complex, multi-step training pipelines, UniGraspTransformer follows a streamlined process: first, dedicated policy networks are trained for individual objects using reinforcement learning to generate successful grasp trajectories; then, these trajectories are distilled into a single, universal network. Our approach enables UniGraspTransformer to scale effectively, incorporating up to 12 self-attention blocks for handling thousands of objects with diverse poses. Additionally, it generalizes well to both idealized and real-world inputs, evaluated in state-based and vision-based settings. Notably, UniGraspTransformer generates a broader range of grasping poses for objects in various shapes and orientations, resulting in more diverse grasp strategies. Experimental results demonstrate significant improvements over state-of-the-art, UniDexGrasp++, across various object categories, achieving success rate gains of 3.5%, 7.7%, and 10.1% on seen objects, unseen objects within seen categories, and completely unseen objects, respectively, in the vision-based setting. Project page: https://dexhand.github.io/UniGraspTransformer.