Learning to Infer Kinematic Hierarchies for Novel Object Instances

TL;DR

This paper introduces a perception system that infers complete kinematic hierarchies of unseen articulated objects from point cloud data, enabling robots to understand and manipulate novel objects without prior templates.

Contribution

It presents a novel neural network-based approach combining point cloud segmentation and graph neural networks to infer kinematic structures of unknown objects.

Findings

Successfully infers kinematic hierarchies from simulated 3D scans.

Demonstrates real-world robotic manipulation using inferred hierarchies.

Achieves accurate prediction of parts and joint types in unseen objects.

Abstract

Manipulating an articulated object requires perceiving itskinematic hierarchy: its parts, how each can move, and howthose motions are coupled. Previous work has explored per-ception for kinematics, but none infers a complete kinematichierarchy on never-before-seen object instances, without relyingon a schema or template. We present a novel perception systemthat achieves this goal. Our system infers the moving parts ofan object and the kinematic couplings that relate them. Toinfer parts, it uses a point cloud instance segmentation neuralnetwork and to infer kinematic hierarchies, it uses a graphneural network to predict the existence, direction, and typeof edges (i.e. joints) that relate the inferred parts. We trainthese networks using simulated scans of synthetic 3D models.We evaluate our system on simulated scans of 3D objects, andwe demonstrate a proof-of-concept use of our system to drivereal-world robotic manipulation.