dGrasp: NeRF-Informed Implicit Grasp Policies with Supervised Optimization Slopes

TL;DR

dGrasp introduces a NeRF-informed implicit grasp policy trained with a novel auxiliary loss that guides the optimization landscape, resulting in improved simulation performance and zero-shot real-world transfer.

Contribution

It proposes a new implicit grasp policy framework with a NeRF-informed value function and a novel auxiliary loss for better optimization landscape shaping.

Findings

Enhanced grasp success rates in simulation

Successful zero-shot transfer to real-world scenarios

Improved optimization landscape for grasp policies

Abstract

We present dGrasp, an implicit grasp policy with an enhanced optimization landscape. This landscape is defined by a NeRF-informed grasp value function. The neural network representing this function is trained on simulated grasp demonstrations. During training, we use an auxiliary loss to guide not only the weight updates of this network but also the update how the slope of the optimization landscape changes. This loss is computed on the demonstrated grasp trajectory and the gradients of the landscape. With second order optimization, we incorporate valuable information from the trajectory as well as facilitate the optimization process of the implicit policy. Experiments demonstrate that employing this auxiliary loss improves policies' performance in simulation as well as their zero-shot transfer to the real-world.