Cross-Category Functional Grasp Transfer

TL;DR

This paper introduces a method for transferring functional grasps across different object categories by leveraging a knowledge graph of similarities, reducing the need for extensive grasp annotations on new objects.

Contribution

It proposes a cross-category grasp transfer approach using a knowledge graph of object similarities, enabling functional grasp synthesis for unseen object categories.

Findings

Knowledge graph improves grasp transfer accuracy

Functional grasps can be synthesized for new object categories

Object similarities facilitate grasp generalization

Abstract

Generating grasps for a dexterous hand often requires numerous grasping annotations. However, annotating high DoF dexterous hand poses is quite challenging. Especially for functional grasps, requiring the hand to grasp the object in a specific pose to facilitate subsequent manipulations. This prompts us to explore how people achieve manipulations on new objects based on past grasp experiences. We find that when grasping new items, people are adept at discovering and leveraging various similarities between objects, including shape, layout, and grasp type. Considering this, we analyze and collect grasp-related similarity relationships among 51 common tool-like object categories and annotate semantic grasp representation for 1768 objects. These objects are connected through similarities to form a knowledge graph, which helps infer our proposed cross-category functional grasp synthesis. Through extensive experiments, we demonstrate that the grasp-related knowledge indeed contributed to achieving functional grasp transfer across unknown or entirely new categories of objects.