23 DoF Grasping Policies from a Raw Point Cloud

TL;DR

This paper introduces a neural geometric approach for high-DoF robotic grasping from partial point clouds, enabling generalization to new objects and stable grasping trajectories.

Contribution

It presents a novel imitation learning method that predicts 23 DoF grasp trajectories directly from point clouds using a geometric-based model of dynamics.

Findings

The policy generalizes well to novel objects.

Combining the policy with a geometric fabric motion planner yields stable grasps.

Ablation studies highlight the importance of object encoding.

Abstract

Coordinating the motion of robots with high degrees of freedom (DoF) to grasp objects gives rise to many challenges. In this paper, we propose a novel imitation learning approach to learn a policy that directly predicts 23 DoF grasp trajectories from a partial point cloud provided by a single, fixed camera. At the core of the approach is a second-order geometric-based model of behavioral dynamics. This Neural Geometric Fabric (NGF) policy predicts accelerations directly in joint space. We show that our policy is capable of generalizing to novel objects, and combine our policy with a geometric fabric motion planner in a loop to generate stable grasping trajectories. We evaluate our approach on a set of three different objects, compare different policy structures, and run ablation studies to understand the importance of different object encodings for policy learning.