Gradient based Grasp Pose Optimization on a NeRF that Approximates Grasp Success

TL;DR

This paper introduces a gradient-based method using NeRF to directly optimize grasp poses for robotic grasping, bypassing explicit object modeling and discretization, and demonstrates promising results in simulation.

Contribution

It presents a novel NeRF-based approach that directly maps grasp poses to success probabilities and optimizes them via gradients, avoiding rendering and discretization limitations.

Findings

Achieves an average translation error of 3mm in simulated grasp tasks.

Generalizes to novel objects in 3DoF grasping scenarios.

Effective in four simulated robotic grasping tasks.

Abstract

Current robotic grasping methods often rely on estimating the pose of the target object, explicitly predicting grasp poses, or implicitly estimating grasp success probabilities. In this work, we propose a novel approach that directly maps gripper poses to their corresponding grasp success values, without considering objectness. Specifically, we leverage a Neural Radiance Field (NeRF) architecture to learn a scene representation and use it to train a grasp success estimator that maps each pose in the robot's task space to a grasp success value. We employ this learned estimator to tune its inputs, i.e., grasp poses, by gradient-based optimization to obtain successful grasp poses. Contrary to other NeRF-based methods which enhance existing grasp pose estimation approaches by relying on NeRF's rendering capabilities or directly estimate grasp poses in a discretized space using NeRF's scene representation capabilities, our approach uniquely sidesteps both the need for rendering and the limitation of discretization. We demonstrate the effectiveness of our approach on four simulated 3DoF (Degree of Freedom) robotic grasping tasks and show that it can generalize to novel objects. Our best model achieves an average translation error of 3mm from valid grasp poses. This work opens the door for future research to apply our approach to higher DoF grasps and real-world scenarios.