Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulation

TL;DR

This paper introduces Local Neural Descriptor Fields (L-NDF), a method enabling robots to generalize manipulation skills to unseen objects by leveraging local geometric features, demonstrated through successful tests in simulation and real-world scenarios.

Contribution

The paper presents L-NDF, a novel approach that uses local geometry-based neural descriptors to transfer manipulation skills to new objects from unseen shape categories.

Findings

Effective transfer of manipulation skills to novel objects

Successful manipulation in both simulation and real-world environments

Leverages local geometry for generalization

Abstract

A robot operating in a household environment will see a wide range of unique and unfamiliar objects. While a system could train on many of these, it is infeasible to predict all the objects a robot will see. In this paper, we present a method to generalize object manipulation skills acquired from a limited number of demonstrations, to novel objects from unseen shape categories. Our approach, Local Neural Descriptor Fields (L-NDF), utilizes neural descriptors defined on the local geometry of the object to effectively transfer manipulation demonstrations to novel objects at test time. In doing so, we leverage the local geometry shared between objects to produce a more general manipulation framework. We illustrate the efficacy of our approach in manipulating novel objects in novel poses -- both in simulation and in the real world.