ShapeGen: Robotic Data Generation for Category-Level Manipulation

TL;DR

ShapeGen is a novel 3D shape generation method that creates diverse, functionally correct manipulation data to improve robotic policy generalization across object categories without requiring real-world data collection.

Contribution

It introduces a two-stage process for generating shape-variated manipulation data using a shape library and minimal human annotation, enhancing in-category policy generalization.

Findings

ShapeGen improves real-world manipulation policy generalization.

Generated data is physically plausible and functionally correct.

Experiments show significant boost in in-category shape diversity handling.

Abstract

Manipulation policies deployed in uncontrolled real-world scenarios are faced with great in-category geometric diversity of everyday objects. In order to function robustly under such variations, policies need to work in a category-level manner, i.e. knowing how to interact with any object in a certain category, instead of only a specific one seen during training. This in-category generalizability is usually nurtured with shape-diversified training data; however, manually collecting such a corpus of data is infeasible due to the requirement of intense human labor and large collections of divergent objects at hand. In this paper, we propose ShapeGen, a data generation method that aims at generating shape-variated manipulation data in a simulator-free and 3D manner. ShapeGen decomposes the process into two stages: Shape Library curation and Function-Aware Generation. In the first stage, we train spatial warpings between shapes mapping points to points that correspond functionally, and aggregate 3D models along with the warpings into a plug-and-play Shape Library. In the second stage, we design a pipeline that, leveraging established Libraries, requires only minimal human annotation to generate physically plausible and functionally correct novel demonstrations. Experiments in the real world demonstrate the effectiveness of ShapeGen to boost policies' in-category shape generalizability. Project page: https://wangyr22.github.io/ShapeGen/.