Towards a Multi-Embodied Grasping Agent

TL;DR

This paper introduces a data-efficient, flow-based grasp synthesis architecture capable of handling various gripper types by exploiting kinematic models, with improved performance and faster inference.

Contribution

It presents a novel equivariant grasping model built from scratch in JAX that generalizes across diverse grippers and scene geometries, using a large-scale dataset.

Findings

Achieved smoother learning and faster inference compared to previous methods.

Successfully generalized grasp synthesis across multiple gripper types.

Utilized a dataset of 25,000 scenes and 20 million grasps.

Abstract

Multi-embodiment grasping focuses on developing approaches that exhibit generalist behavior across diverse gripper designs. Existing methods often learn the kinematic structure of the robot implicitly and face challenges due to the difficulty of sourcing the required large-scale data. In this work, we present a data-efficient, flow-based, equivariant grasp synthesis architecture that can handle different gripper types with variable degrees of freedom and successfully exploit the underlying kinematic model, deducing all necessary information solely from the gripper and scene geometry. Unlike previous equivariant grasping methods, we translated all modules from the ground up to JAX and provide a model with batching capabilities over scenes, grippers, and grasps, resulting in smoother learning, improved performance and faster inference time. Our dataset encompasses grippers ranging from humanoid hands to parallel yaw grippers and includes 25,000 scenes and 20 million grasps.