Generative 3D Part Assembly via Dynamic Graph Learning

TL;DR

This paper introduces a dynamic graph learning framework for estimating 6-DoF poses of 3D parts to assemble shapes, using iterative refinement and relational reasoning, advancing autonomous 3D part assembly.

Contribution

It presents a novel assembly-oriented dynamic graph neural network that performs sequential coarse-to-fine pose refinement for 3D part assembly tasks.

Findings

Outperforms three strong baseline methods in pose estimation accuracy.

Effectively models relational reasoning between parts during assembly.

Demonstrates robustness across diverse 3D shape datasets.

Abstract

Autonomous part assembly is a challenging yet crucial task in 3D computer vision and robotics. Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation. In this paper, we focus on the pose estimation subproblem from the vision side involving geometric and relational reasoning over the input part geometry. Essentially, the task of generative 3D part assembly is to predict a 6-DoF part pose, including a rigid rotation and translation, for each input part that assembles a single 3D shape as the final output. To tackle this problem, we propose an assembly-oriented dynamic graph learning framework that leverages an iterative graph neural network as a backbone. It explicitly conducts sequential part assembly refinements in a coarse-to-fine manner, exploits a pair of part relation reasoning module and part aggregation module for dynamically adjusting both part features and their relations in the part graph. We conduct extensive experiments and quantitative comparisons to three strong baseline methods, demonstrating the effectiveness of the proposed approach.