AssemblyBench: Physics-Aware Assembly of Complex Industrial Objects

TL;DR

AssemblyBench is a new synthetic dataset and AssemblyDyno is a transformer-based model for physically plausible industrial object assembly, improving pose estimation and trajectory prediction.

Contribution

The paper introduces AssemblyBench, a comprehensive dataset, and AssemblyDyno, a novel transformer model for complex industrial assembly tasks.

Findings

AssemblyDyno outperforms prior methods in pose estimation.

AssemblyDyno achieves higher trajectory feasibility in physics-based simulations.

AssemblyBench captures complex industrial assembly scenarios.

Abstract

Assembling objects from parts requires understanding multimodal instructions, linking them to 3D components, and predicting physically plausible 6-DoF motions for each assembly step. Existing datasets focus on simplified scenarios, overlooking shape complexities and assembly trajectories in industrial assemblies. We introduce AssemblyBench, a synthetic dataset of 2,789 industrial objects with multimodal instruction manuals, corresponding 3D part models, and part assembly trajectories. We also propose a transformer-based model, AssemblyDyno, which uses the instructional manual and the 3D shape of each part to jointly predict assembly order and part assembly trajectories. AssemblyDyno outperforms prior works in both assembly pose estimation and trajectory feasibility, where the latter is evaluated by our physics-based simulations.