OmniHang: Learning to Hang Arbitrary Objects using Contact Point Correspondences and Neural Collision Estimation

TL;DR

This paper presents OmniHang, a system enabling robots to learn and execute the task of hanging arbitrary objects on diverse supports by estimating contact points, refining poses with reinforcement learning, and planning collision-free paths.

Contribution

The paper introduces a novel approach combining contact point correspondence estimation, deep reinforcement learning for pose refinement, and neural collision estimation for path planning in object hanging tasks.

Findings

Achieves 68.3% success rate in stable pose prediction.

Attains 52.1% F1 score in feasible path finding.

Develops a large-scale synthetic dataset for training and evaluation.

Abstract

In this paper, we explore whether a robot can learn to hang arbitrary objects onto a diverse set of supporting items such as racks or hooks. Endowing robots with such an ability has applications in many domains such as domestic services, logistics, or manufacturing. Yet, it is a challenging manipulation task due to the large diversity of geometry and topology of everyday objects. In this paper, we propose a system that takes partial point clouds of an object and a supporting item as input and learns to decide where and how to hang the object stably. Our system learns to estimate the contact point correspondences between the object and supporting item to get an estimated stable pose. We then run a deep reinforcement learning algorithm to refine the predicted stable pose. Then, the robot needs to find a collision-free path to move the object from its initial pose to stable hanging pose. To this end, we train a neural network based collision estimator that takes as input partial point clouds of the object and supporting item. We generate a new and challenging, large-scale, synthetic dataset annotated with stable poses of objects hung on various supporting items and their contact point correspondences. In this dataset, we show that our system is able to achieve a 68.3% success rate of predicting stable object poses and has a 52.1% F1 score in terms of finding feasible paths. Supplemental material and videos are available on our project webpage.