Omnigrasp: Grasping Diverse Objects with Simulated Humanoids

TL;DR

This paper introduces Omnigrasp, a method enabling simulated humanoids to grasp and transport over 1200 diverse objects along arbitrary trajectories, using a scalable, human-like control approach that generalizes well to unseen objects.

Contribution

The paper presents a novel humanoid control method that learns to grasp and move a wide variety of objects without requiring paired motion data, improving scalability and generalization.

Findings

Achieves state-of-the-art success rates in trajectory following.

Successfully generalizes to unseen objects.

Scales to over 1200 object types with simple training setup.

Abstract

We present a method for controlling a simulated humanoid to grasp an object and move it to follow an object's trajectory. Due to the challenges in controlling a humanoid with dexterous hands, prior methods often use a disembodied hand and only consider vertical lifts or short trajectories. This limited scope hampers their applicability for object manipulation required for animation and simulation. To close this gap, we learn a controller that can pick up a large number (>1200) of objects and carry them to follow randomly generated trajectories. Our key insight is to leverage a humanoid motion representation that provides human-like motor skills and significantly speeds up training. Using only simplistic reward, state, and object representations, our method shows favorable scalability on diverse objects and trajectories. For training, we do not need a dataset of paired full-body motion and object trajectories. At test time, we only require the object mesh and desired trajectories for grasping and transporting. To demonstrate the capabilities of our method, we show state-of-the-art success rates in following object trajectories and generalizing to unseen objects. Code and models will be released.